Simplex Method

I. Summary of the Simplex Method

A. Add slack variables to change the constraints into equations and write all variables to
the left of the equal sign and constants to the right.
B. Write the objective function with all nonzero terms to the left of the equal sign and
zero to the right. The variable to be maximized must be positive.
C. Set up the initial simplex tableau by creating an augmented matrix from the
equations, placing the equation for the objective function last.
D. Determine a pivot element and use matrix row operations to convert the column
containing the pivot element into a unit column.
E. If negative elements still exist in the bottom row, repeat Step 4. If all elements in
the bottom row are positive, the process has been completed.
F. When the final matrix has been obtained, determine the final basic solution. This
will give the maximum value for the objective function and the values of the
variables where this maximum occurs.

II. Example of the Simplex Method
The Ace Novelty Company has determined that the profits are $6, $5, and $4 for each
type-A, type-B, and type-C souvenir that it plans to produce. To manufacture a type-A
souvenir requires 2 minutes on machine I, 1 minute on machine II, and 2 minutes on
machine III. A type-B souvenir requires 1 minute on Machine I, 3 minutes on machine II,
and 1 minute on machine III. A type-C souvenir requires 1 minutes on machine I and 2
minutes on each of machines II and III. There are 3 hours available on machine I, 5 hours
available on machine II, and 4 hours available on machine III for manufacturing these
souvenirs each day. How many souvenirs of each type should Ace Novelty make per day
in order to maximize its profit?

I. Organize the information












souvenir
machine
A
B
C
maximum
available

number
of units
x
y
z
I
(min)

2
1
1
180
II
(min)

1
3
2
300
III
(min)

2
1
2
240
profit in
$/unit
6
5
4
P
II. Write the constraints and objective function

2x + y + z 180
1. The constraints are x + 3y + 2z 300
2x + y +2z 240
x 0, y 0, z 0

2. The objective function is P = 6x + 5y + 4z, which is to be maximized.

III. Insert slack variables to change inequalities into equations; rewrite objective
function

2x + y + z + s = 180
The equations are x + 3y + 2z + t = 300
2x + y +2z + u = 240
-6x - 5y - 2z + P = 0

IV. Write the initial simplex tableau

x y z s t u p constant
2 1 1 1 0 0 0 180
1 3 2 0 1 0 0 300
2 1 2 0 0 1 0 240
-6 -5 -4 0 0 0 1 0

1. The basic feasible solution from the initial tableau is x = 0, y = 0, z = 0, s = 180, t = 300,
u = 240, and P = 0.

2. The interpretation of the solution is that if no souvenirs of any type are made, there
will be no profit!

3. Since there are negative indicators below the line, the objective function has not been
maximized.

V. Select the pivot element
1. Select the column with the most negative indicator: column 1 in this tableau.

2. Divide each constant to the right of the bar by the corresponding (nonzero) element in the
pivot column.
180/2 = 90 select the smallest quotient
300/1 = 300
240/2 = 120

3. The pivot element is the intersection of the column with the most negative indicator and
the row with the smallest quotient. The pivot is the 2 in column 1 in this tableau.

VI. Change the pivot element to 1

x y z s t u p constant
1 .5 .5 .5 0 0 0 90 R
1
R
1

1 3 2 0 1 0 0 300
2 1 2 0 0 1 0 240
-6 -5 -4 0 0 0 1 0


VII. Pivot about the pivot element

x y z s t u p constant
1 .5 .5 .5 0 0 0 90
0 2.5 1.5 -.5 1 0 0 210 R
1
+R
2
R
2

0 0 1 -1 0 1 0 60 2R
1
+R
3
R
3

0 -2 -1 3 0 0 1 540 6R
1
+R
4
R
4


1. The basic feasible solution from this tableau is x = 90, y = 0, z = 0, s = 0, t = 210, u = 60,
and P = 540.

2. The interpretation of the solution is that if 90 souvenirs of type A are made and none of
the other two types, there will a profit of $540.

3. This is a definite improvement over the initial tableau, but since there are still negative
indicators below the line, the objective function has not been maximized.

VIII. Determine a new pivot element
1. Select the column with the most negative indicator: column 2 in this tableau.

2. Divide each constant to the right of the bar by the corresponding (nonzero)
element in the pivot column.
90/.5 = 180
210/2.5 = 84 select the smallest quotient

3. The pivot element is the intersection of the column with the most negative indicator and
the row with the smallest quotient. The pivot is the 2.5 in column 2 in this tableau.
IX. Change the pivot element to 1

x y z s t u p constant
1 .5 .5 .5 0 0 0 90
0 1 .6 -.2 .4 0 0 84 1/2.5 R
2
R
2

0 0 1 -1 0 1 0 60
0 -2 -1 3 0 0 1 540


X. Pivot about the pivot element

x y z s t u p constant
1 .5 .5 .5 0 0 0 90 .5R
2
+R
1
R
1

0 1 .6 -.2 .4 0 0 84
0 0 1 -1 0 1 0 60
0 -2 -1 3 0 0 1 540 2R
2
+R
4
R
4

1. The basic feasible solution from this tableau is x = 48, y= 84, z = 0, s = 0, t = 0, u = 60,
and P = 708.

2. The interpretation of the solution is that if 48 souvenirs of type A, 84 of type B, and none
of type C are made, there will be a profit of $708.

3. Since there are no negative indicators below the line, the objective function has been
maximized.









Simple Method in Matrix Form, and Sensitivity Analysis

I. Matrix Form of Simplex Method
Consider the standard form max-LP given in matrix-form:
max z = c
T
x
s.t. Ax = b (1)
x 0.

The necessary slack and surplus/artificial variables have already been added (and
hence we have all constraints in the = form). Here, A is an m n matrix (i.e., m rows
and n columns), b is an m-vector, c and x are n-vectors. Thus, the n variable
(represented as x) includes the slack/surplus/artificial variables.

Suppose we know which of the n variables are basic in the optimal tableau. Since
there are m constraints, there will be m basic variables and (n m) non-basic
variables. Grouping all the basic variables together (and all the non-basic variables
together), we split the vector x into , where x
B
denotes the m basic


variables, and x
N
the non-basic variables. Subsequently, we also group the columns
of A corresponding to the variables x
B
into the mm matrix B, and the remaining
non-basic columns into the m (n m) matrix N. The cost vector is similarly divided
as c
T
= [c
T
B
c
T
N
]. Thus, the starting tableau for the simplex method can be written as
follows.

z x
B
x
N
rhs
1 -c
T
B
-c
T
N
0 (2)
0 B N b

Notice here that 0 is the m-vector of all zeros. Applying the simplex method, the
optimal tableau looks something like what is shown below:

z x
B
x
N
rhs
1 0 ? ? (3)
0 I ? ?

We know that the basic variables form the canonical columns in the optimal tableau,
thus giving the identity matrix I as shown. The important task of course is to figure
out what each of the ?s are. Recall that the steps of the simplex method are
equivalent to those of the Gauss-Jordan method (we are performing elementary row
operations in both cases). Hence, we can think of obtaining the optimal tableau by
multiplying the initial tableau by a suitable matrix on the left (which is the
transformation matrix representing all the EROs). Let us call this transformation
x
B_

x
N


matrix as T (notice that it will be an (m+1)(m+1) matrix). If you concentrate on the
columns of z and x
B
, you can see that multiplying by T actually converts this part of
the tableau to the identity matrix:


T = (4)


Hence, we must have that

T = = (5)



You should check that the inverse of T is indeed as given here. The multiplication
looks similar to the multiplication of 22 matrices, except that the elements
themselves are matrices or vectors. Knowing T, we can write the optimal tableau as
follows.


=


Thus, the optimal objective function value is given by z_ = cT BB1b, and the
optimal solution is given by x
B
= B
1
b, x
N
= 0. We will use the Farmer Jones example
to illustrate the various cases of sensitivity analysis that we want to study. To make
the example interesting, the objective function coefficient of acres of wheat (
x2
) will
be set at 25 (as opposed to the original value of 100 think about the price per bushel
of wheat being set at $1 in place of $4). The LP is given below.

max z = 30x
1
+ 25x
2
(total revenue)
s.t. x
1
+ x
2
7 (total acres)
4x
1
+ 10x
2
40 (labor hrs)

x
1
3 (min corn)

x
1
, x
2
0 (non-neg)

Notice that the (min corn) constraint has been divided by 10 throughout. The starting
tableau and the optimal tableau for the above LP are the following.

Table 1: Starting tableau
z x
1
x
2
s
1
s
2
e
3
a
3
rhs
1 -30 -25 0 0 0 M 0
0 1 1 1 0 0 0 7
0 4 10 0 1 0 0 40
0 1 0 0 0 -1 1 3

1 0 -c
T
N
+c
T
B
B
-1
N c
T
B
B
-1
b
0 I B
-1
N
B
-1
b
1 -c
T
B
-c
T
N
0
0 B N b
-1
1 -c
T
B

0 B
1 0
0 I
1 -c
T
B

0 B
1 c
T
B
B
-1

0 B
-1
1 c
T
B
B
-1

0 B
-1
Table 2: Optimal tableau
z x
1
x
2
s
1
s
2
e
3
a
3
rhs
1 0 5 30 0 0 M 240
0 0 1 1 0 1 -1 4
0 0 6 -4 1 0 0 12
0 1 1 1 0 0 0 7

The variables that are basic in the optimal tableau are x
T
B
= [e
3
s
2
x
1
], with the
optimal solution given by x
1
= 7, s
2
= 12, e
3
= 4, and z* = 210. Hence, c
T
B
= [0 0 30],
and the matrix B and its inverse are as given below.


B = , and B
-1
=


Check the inverse of B. In fact, we can read of B
1
from the optimal tableau! Note
that the element in the right-bottom position of T is B
1
. Hence, if some of the
columns in the initial tableau had the identity matrix I in the rows 1 to m, then the
same columns will have B
1
in the optimal tableau. For example, if the non-basic part
of A were in fact identity, i.e., N = I, then in the optimal tableau, we will have (under
the columns of x
N
) B
1
N = B
1
I = B
1
.
At the same time, the slack and artificial variables indeed form the identity matrix in
the initial tableau (before converting it to canonical form) see the columns of s
1
, s
2
,
a
3
in the initial tableau (Table 1). Hence, B
1
can be read off from the columns of s
1
,
s
2
, and a
3
from the optimal tableau (check to make sure that this observation indeed is
true).

II. Sensitivity Analysis
We are now ready to consider the various cases of sensitivity analysis. The optimal
tableau will remain optimal as long as

1. c
T
N
+ c
T
B
B
1
N 0 (for a max-LP, all entries in the z-row should be non-negative
for optimality), and
2. B
1
b 0 (in order to maintain feasibility).

We will use these two conditions to check whether the current basis still remains
optimal after one (or more) of the parameters is (are) changed.

2.1 Changing c
j
when x
j
is non-basic

Here, x
j
is one of the variables in x
N
. We denote the column of N corresponding to x
j

by
aj
. Consider changing c
j
to c
j
+ . From Equation (6), we can see that the only
entry in the optimal tableau that possibly changes is the x
j
-entry in Row-0. As seen
from the optimal tableau for the Farmer Jones problem, x
2
is a non-basic variable.
Consider changing the objective function coefficient of x
2
from 25 to 25 + (you
could consider the yield per acre of wheat changing from 25 to 25 + ). Our goal is to
0 0 1
0 1 4
-1 0 1
1 0 -1
-4 1 0
1 0 0
find the range of values of for which the current basis still remains optimal. The
new entry for x
2
in Row-0 of the optimal tableau is

2
= -(25 + ) + c
T
B
B
-1
a
2
= (-25 - ) + [0 0 30]



= (-25 - ) + [30 0 0] = 5 - .


Notice that a
2
is the column of the matrix A corresponding to x
2
. We need c
2
0 for
the current basis to remain optimal (condition (1)). Hence, the range of values of
for which the current basis remains optimal is 5. Notice that the optimal solution
and the optimal objective function value do not change when is in this range. Also,
recall that the optimal solution was to farm corn in all the 7 acres available (x
1
= 7, x
2

= 0 in the optimal solution). With 5, the revenue per acre of wheat is 30 (which
is the value for corn). At the same time, each acre of wheat requires more hours of
labor (10) than an acre of corn (4). There is no minimum level of wheat production
required either. Hence, it makes sense not to farm any wheat. If the revenue per acre
of wheat becomes higher than that of corn (> 30), one should farm some acres of
wheat in order to make the maximum revenue. This value, 5, is called the reduced
cost of the non-basic variable x
2
. The reduced cost of a non-basic variable (in the
optimal tableau) of a max-LP is defined as the maximum amount by which its
objective function coefficient can increase while the current basis still remains
optimal. Hence, if the objective function coefficient of wheat changes to 32 (i.e., =
7), the current basis is no longer optimal in fact, it is sub-optimal in the sense that
the value of x
2
can be increased from 0 to some positive value and the value of z* can
be increased. In this case, in order to find the new optimal tableau, one will have to
perform one more simplex pivot (with x
2
as the entering variable).

2.2 Changing c
j
when x
j
is basic

Here, x
j
is one of the variables in x
B
. Consider changing the objective function
coefficient of x
1
from 30 to 30 + . c
T
B
also changes here c
T
B
= [0 0 30 + ]. Since
c
B
changes, all the entries in Row-0 corresponding to x
N
(i.e., all the non-basic
variables) could possibly change. The new entries in Row-0 under x
N
are given by

T
N
= -c
T
N
+ c
T
B
B
-1
N = [-25 0 M] + [0 0 30 + ]


= [-25 0 M] + [30 + 30 + 0] = [5 + 30 + M].

Using the condition (1) again, the current basis remains optimal if all entries in cN
are 0, i.e., if 5 + 0 and 30 + 0. Hence, the range of values of for which the
1 0 -1 1
-4 1 0 10
1 0 0 0
1
10
0
1 0 -1 1 1 0
-4 1 0 10 0 0
1 0 0 0 0 1
current basis remains optimal is 5. For this range, the revenue per acre of corn
will be at least as big as the value for wheat (25), and hence one will not grow any
wheat. If = 8 for example, then each acre of wheat will yield a higher revenue than
an acre of corn (25 and 22 respectively). The current basis becomes sub-optimal in
this case. In order to find the new optimal solution, one will have to pivot x
2
into the
basis (perform one more simplex pivot).

2.3 Changing the right-hand-side (bi) of the i-th constraint

Consider changing b
i
to b
i
+ . From Equation (6), we can see that the entries in the
rhs (last) column will change. For example, consider changing the rhs of acres
constraint to 7 + (i = 1, i.e., the first constraint here). This change models the
situation where Farmer Jones has some extra land to farm.


The new rhs vector can be written as = . Hence,



B
-1
= B
-1
b +



Using the condition (2), in order for the current basis to still remain optimal, we need
all elements of x
B
0. In other words, 4 + 0, 12 4 0, and 7+ 0, which
give 4, 12/4 = 3, 7, or 4 3. The current basis remains optimal as
long as is in this range. Notice that, when > 3, Jones will have more than 10 acres
to farm, but could only use 10 due to the limit on the labor hours (why?). Thus the
optimal solution will change when = 4, for instance. Similarly, when = 4.5, the
minimum level of corn cannot be satisfied (as Jones will have only 2.5 acres), and
hence the current basis will no longer be optimal.
For 4 3, the new optimal objective function is given by (using the expression
for B
1
b from Equation (7))


* = c
T
B

B
= c
T
B
B
-1
b = [0 0 30] = 210 + 30


Hence the shadow price of the acres constraint is 30. Recall that the shadow price
of a constraint is defined as the improvement (increase for a max-LP) in the objective
function value for a unit increase the rhs value of the constraint.

2.4 Changing the column (both c
j
and a
j
) when x
j
is non-basic

The general method outlined above can also be used to calculate the new optimal
tableau (and optimal basis) when more than one parameter is changed. For example,
7 + 7
40 = 40 + 0
3 3 0
1 0 -1 4 4 +
-4 1 0 0 = 12 + -4 = 12 - 4 =
B
. (7)
1 0 0 0 7 7 +
4 +
12 - 4
7 +
consider changing the objective function coefficient of x
2
(which is non-basic) from
25 to 35, and at the same time, changing its coefficient in the labor hours constraint
from 10 to 8 (i.e., the number of labor hours required for an acre of wheat is now 8).
Using the formulae given in Equation (6), the new entries in the column of x
2
are
given by









Notice that we used the result for c
T
B
B
1
= [30 0 0], which we already saw in section
2.1. Since the modified entry in the Row-0 under x
2
is no longer non-negative (it is
5), the current basis is no longer optimal. Intuitively, it required less number of labor
hours and generates more revenue to farm an acre of wheat after these changes. In
order to find the new optimal tableau, the column of x
2
in the original optimal tableau
should be replaced with the one derived above, and one more simplex pivot should be
performed. The new optimal solution thus obtained is e
3
= 1, x
2
= 4, x
1
= 3, with z* =
225.













1
-35 + [30 0 0] 0 -5
-c
2
+ c
T
B
B
-1
a
2
0
= =
1


B
-1
a
2
1 0 -1 1 4
-4 1 0 8 1
1 0 0 0
Duality
To every linear program there is a dual linear program with which it is intimately connected. We
rst state this duality for the standard programs. As in Section 1, c and x are n-vectors, b and y
are m-vectors, and A is an m n matrix. We assume m 1 and n 1.
Denition. The dual of the standard maximum problem.
*maximize c
T
x
subject to the constraints Ax b and x 0
is dened to be the standard minimum problem
*minimize y
T
b
subject to the constraints y
T
A c
T
and y 0
Let us reconsider the numerical example of the previous section: Find x
1
and x
2
to maximize x
1
+
x
2
subject to the constraints x
1
0, x
2
0, and
x
1
+ 2x
2
4
4x
1
+ 2x
2
12
x
1
+ x
2
1.
The dual of this standard maximum problem is therefore the standard minimum problem: Find
y
1
, y
2
, and y
3
to minimize 4y
1
+ 12y
2
+y
3
subject to the constraints y
1
0, y
2
0, y
3
0, and
y
1
+ 4y
2
y
3
1
2y
1
+ 2y
2
+ y
1

If the standard minimum problem (2) is transformed into a standard maximum problem (by
multiplying A, b, and c by 1 ), its dual by the denition above is a standard minimum problem
which, when transformed to a standard maximum problem (again by changing the signs of all
coecients) becomes exactly (1). Therefore, the dual of the standard minimum problem (2) is
the standard maximum problem (1). The problems (1) and (2) are said to be duals.

Theorem 1. If x is feasible for the standard maximum problem (1) and if y is feasible for its dual
(2), then
c
T
x y
T
b.
Proof.
c
T
x y
T
Ax y
T

The rst inequality follows from x 0 and c
T
y
T
A. The second inequality follows from y 0
and Ax b.
Corollary 1. If a standard problem and its dual are both feasible, then both are bounded feasible.
Proof. If y is feasible for the minimum problem, then (7) shows that y
T
b is an upper bound for
the values of c
T
x for x feasible for the maximum problem. Similarly for the converse.
Corollary 2. If there exists feasible x and y for a standard maximum problem (1) and its dual
(2) such that c
T
x = y
T
b, then both are optimal for their respective problems.
Proof. If x is any feasible vector for (1), then c
T
x y
T
b = cTx, which shows that x
is optimal. A symmetric argument works for y.

The following fundamental theorem completes the relationship between a standard problem and
its dual. It states that the hypothesis of Corollary 2 is always satised if one of the problems is
bounded feasible. The proof of this theorem is not as easy as the previous theorem and its
corollaries. We postpone the proof until later when we give a constructive proof via the simplex
method. (The simplex method is an algorithmic method for solving linear programming
problems.) We shall also see later that this theorem contains the Minimax Theorem for nite
games of Game Theory.

The Duality Theorem.
If a standard linear programming is bounded feasible, then so is its dual, their values are equal,
and there exists optimal vector for both problems.

Interpretation of the Dual.
In addition to the help it provides in finding a solution, the dual problem offers advantages in the
interpretation of the original, primal problem.

Example:
Original Problem Dual
Minimize: z = 8.5 x
1
+4x
2
Maximize: z = 8y
1
+ 12y
2
+ 9y
3

Constraints: 2x
1
+ x
2
8 Constraints: 2y
1
+ 6y
2
+ y
3
8.5
6x
1
+ x
2
12 y
1
+ y
2
+ 3y
3
4
x
1
+ 3x
2
9 y
1
+ y
2
0
x
1
, x
2
0

Standard form
8y
1
+ 12y
2
+ 9y
3
+ 0s
1
+0s
2

2y
1
+ 6y
2
+ y
3
+ 1s
1
+ 0s
2
= 8.5
y
1
+ y
2
+3y
3
+ 0s
1
+ 1s
2
= 4

Matrix form

2 6 1 1 0
y
1
+ y
2
+ y
3
+ s
1
+ s
2
= P
0

1 1 3 0 1