1.

Basics on Linear Programming

Javier Larrosa
Albert Oliveras
Enric Rodrguez-Carbonell
Problem Solving and Constraint Programming (RPAR)

Session 1 p.1/27

Linear Programs (LPs)

A linear program is an optimization problem of the form
min cT x
A 1 x b1
A 2 x = b2
A 3 x b3
x Rn
c Rn , bi Rmi , Ai Rnmi , i = 1, 2, 3
x is the vector of variables
cT x is the cost or objective function
A1 x b1 , A2 x = b2 and A3 x b3 are the constraints
Session 1 p.2/27

Notes on the Definition of LP

Solving minimization or maximization is equivalent:
max{ f (x) | x S } = min{ f (x) | x S }

Satisfiability problems are a particular case:

take arbitrary cost function, e.g., c = 0

Session 1 p.3/27

Equivalent Forms of LPs (1)

This form is not the most convenient for algorithms
WLOG we can transform such a problem as follows
1. Split = constraints into and constraints
min cT x

min cT x
A 1 x b1
A 2 x = b2
A 3 x b3

A 1 x b1
=

A 2 x b2
A 2 x b2
A 3 x b3

Now all constraints are or

Session 1 p.4/27

Equivalent Forms of LPs (2)

2. Transform constraints into constraints
by multiplying by -1
min cT x
A 1 x b1
A 2 x b2

min cT x
=

A 1 x b1
A2 x b2

Now all constraints are

Session 1 p.5/27

Equivalent Forms of LPs (3)

3. Replace variables x by y z , where y , z are vectors of
fresh variables, and add constraints y 0, z 0
min cT x
Ax b

min cT y cT z
=

Ay Az b
y, z 0

Now all constraints are and all variables have to be 0

Session 1 p.6/27

Equivalent Forms of LPs (4)

4. Add a slack variable to each constraint to convert it
into =
min cT x

min cT x
Ax b
x0

Ax + s = b
x, s 0

Now all constraints are = and all variables have to be 0

Session 1 p.7/27

Equivalent Forms of LPs (5)

Example:
min x 2y

min x 2y
x+y 3
0x2
0y2

x + y + s1 = 3
=

x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

Session 1 p.8/27

Equivalent Forms of LPs (6)

In the end we get
min cT x
Ax = b
x0
c Rn , b Rm , A Rnm , n m, rank(A) = m

These transformations are not strictly necessary

(they increase no. of constraints and variables),
but are convenient in a first formulation of the algorithms
Often variables are identified with columns of the matrix,
and constraints are identified with rows
Session 1 p.9/27

Methods for Solving LPs

Simplex algorithms
Interior-point algorithms

Session 1 p.10/27

Methods for Solving LPs

Simplex algorithms
Interior-point algorithms

Session 1 p.10/27

Basic Definitions (1)

min cT x
Ax = b
x0

Any vector x such that Ax = b is called a solution

A solution x satisfying x 0 is called a feasible solution
An LP with feasible solutions is called feasible;
otherwise it is called infeasible
A feasible solution x is called optimal
if cT x cT x for all feasible solution x
A feasible LP with no optimal solution is unbounded
Session 1 p.11/27

Basic Definitions (2)

min x 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0
(x, y, s1 , s2 , s3 ) = (1, 1, 5, 3, 3) is solution but not feasible
(x, y, s1 , s2 , s3 ) = (1, 1, 1, 1, 1) is a feasible solution

Session 1 p.12/27

Basic Definitions (3)

min x y
x + y + s1 =
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

If = 1 the LP is not feasible

If = 3, = 2 then
(x, y, s1 , s2 , s3 ) = (1, 2, 0, 1, 0) is the only optimal solution

There may be more than one optimal solution:

If = 3 and = 1 then
{(1, 2, 0, 1, 0), (2, 1, 0, 0, 1), ( 32 , 23 , 0, 12 , 21 )} are optimal
Session 1 p.13/27

Basic Definitions (4)

y
min x 2y
min x y

y2

x+y 3

(1, 2)
(2, 1)

x0
y0

x2
x

Session 1 p.14/27

Basic Definitions (5)

y
y2

min x + 2y

x+y 3

x+y 3
0x2
y2

x0

x2
x

Unbounded LP

min x + 2y

Session 1 p.15/27

Basic Definitions (6)

y
min x 2y

min x 2y
y2

x+y 3
0x2

(1, 2)

x+y 3

y2

LP is bounded,
but set of feasible
solutions is not

x0

x2
x

Session 1 p.16/27

Bases (1)
Let us denote by a1 , ..., an the columns of A
A matrix of m columns (ak1 , ..., akm ) is a basis
if the columns are linearly independent
If (ak1 , ..., akm ) is a basis,
then the variables (xk1 , ..., xkm ) are called basic
We usually denote
by B the list of indices (k1 , ..., km ), and
by R the list of indices (1, 2, ..., n) B; and
by B the matrix (ai | i B), and
by R the matrix (ai | i R)
xB the basic variables, xR the non-basic ones
Session 1 p.17/27

Bases (2)
min x 2y

x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

1 1 1 0 0

A= 1 0 0 1 0
0 1 0 0 1

(s1 , s2 , x) do not form a basis

(s1 , s2 , s3 ) form a basis, where xB = (s1 , s2 , s3 ), xR = (x, y)

1 0 0

B= 0 1 0
0 0 1

1 1

R= 1 0
0 1

Session 1 p.18/27

Bases (3)
If B is a basis, then the following holds
BxB + RxR = b

Hence:
xB = B 1 b B 1 RxR

Non-basic variables determine values of basic ones

If non-basic variables are set to 0, we get the solution
xR = 0, xB = B 1 b

Such a solution is called a basic solution

If a basic solution satisfies xB 0 then it is called a
basic feasible solution, and the basis is feasible
Session 1 p.19/27

Bases (4)
Basis (s1 , s2 , s3 ) is feasible
min x 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

s1 = 3 x y
s2 = 2 x

s3 = 2 y

1 0 0

B= 0 1 0
0 0 1

B = 2
2

1 1

R= 1 0
0 1

R =

0
0

Session 1 p.20/27

Bases (5)
Basis (x, y, s1 ) is not feasible
min x 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

x = 2 s2
y = 2 s3

s1 = 1 + s2 + s3

1 1 1

B= 1 0 0
0 1 0

B = 2
1

0 0

R= 1 0
0 1

R =

0
0

Session 1 p.21/27

Bases (6)
A basis is called degenerate when at least one component
of its basic solution xB is null
min x 2y

x + y + s1 = 4
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

1 1 0

R= 0 0
0 1

B= 1 0 1
0 1 0

x = 2 + s3 s1
y = 2 s3

s =s s
2
1
3

1 0

B = 2
0

Session 1 p.22/27

Geometry of LPs (1)

Set of feasible solutions of a LP is a convex polyhedron
Basic feasible solutions are
vertices of the convex polyhedron

Session 1 p.23/27

Geometry of LPs (2)

min x 2y
x + y + s1 = 3
x + s2 = 2
y + s3 = 2
x, y, s1 , s2 , s3 0

y
y2
(0, 2) 1

x+y 3

xB1 = (y, s1 , s2 )
xB2 = (x, y, s2 )
xB3 = (x, y, s3 )
xB4 = (x, s1 , s3 )

(1, 2)

(2, 1)

x0

x2
5

(0, 0)

y0

(2, 0)

xB5 = (s1 , s2 , s3 )
Session 1 p.24/27

Geometry of LPs (3)

Th. (Minkowski-Weyl)
Let P be a LP.
A point x is a feasible solution to P
iff
there exist basic feasible solutions v1 , ..., vr Rn and
vectors r1 , ..., rs Rn such that
x=

r
X

i v i +

i=1

for certain i , j such that

s
X

j rj

j=1

Pr

i=1 i

= 1 and i , j 0.
Session 1 p.25/27

Possible Outcomes of a LP (1)

Th. (Fundamental Theorem of Linear Programming)
Let P be a LP.
Then exactly one of the following holds:
1. P is infeasible
2. P is unbounded
3. P has an optimal basic feasible solution
It is sufficient to investigate basic feasible solutions!

Session 1 p.26/27

Possible Outcomes of a LP (2)

Proof: Assume P feasible and with optimal solution x .
By Minkowski-Weyl theorem, we can write
Ps
Pr
T
T
T
c x =

where

Pr

i=1 i

i=1 i c

vi +

j=1 j c

rj

= 1 and i , j 0.

If there is j such that cT rj < 0 then objective value

can be decreased by taking j larger. Contradiction!
Otherwise cT rj 0 for all j . Assume cT x < cT vi for all i.
Pr
Pr
T
T
T
T
c x

i=1 i c

vi >

i=1 i c

x =c x

Contradiction! Thus there is i such that cT x cT vi ;

in fact, cT x = cT vi by the optimality of x .

Session 1 p.27/27