Beruflich Dokumente
Kultur Dokumente
Fall 2000
1
Table of Contents
1. Background 1
2. Introduction to Linear Programming 2
2.1 Basic Forms 2
2.2 Modeling 4
3. Solving Linear Programs 7
3.1 Geometry and Basic Notions 8
3.1.1 The Feasible Region of Linear Programs 10
3.1.2 Extreme Points 12
3.1.3 Basic Solutions 13
3.2 The Simplex Algorithm 18
4. Generalizations 25
4.1 Artificial Variables 25
4.1.1 Big-M 27
4.1.2 Two-Phase Method 29
4.2 Empty Solution Space 32
4.3 Alternative Optima 34
4.4 Unbounded Solution Spaces 36
4.5 Degeneracy 37
5. Final Comments 41
6. Exercises 43
7. References 49
2
LINEAR PROGRAMMING
1 Background
In this document, we will look at a class of optimization problems that are
quite well-solved in the sense that especially powerful mathematical and com-
putational machinery exists for dealing with them. Before we begin, however,
let us examine what we mean by a mathematical optimization problem in gen-
eral. We can state this in the following way:
P: max(min) z(x1 , x2 , . . . , xn )
s.t. x1 x2 x3 ≤ 10
x1 + x22 + x3
2
≤ 12.
1
max z = x31 + x2 x3 x4 + x22 − x3 (x2 − x4 )2
s.t. x1 + x2 + x3 + x4 = 7
PC : max z = c1 x1 + c2 x2 + . . . + cn xn
x1 , x2 , . . . , xn ≥ 0.
2
The last, nonnegativity constraints require that all variables take on values
no less than 0; the cj , aij , and bj parameters are problem coefficients. Of-
ten the bj values are called the right-hand-sides of the formulation. Note
also that the model indicated by PC is referred to as canonical form, i.e.,
objective function of “maximize” form, less-than-or-equal inequalities, and
nonnegativity restrictions on all of the variables. Readers may want to take
time to convince themselves that every linear program can be expressed in
canonical form.
Often, it is convenient to consider an LP problem in compact form. For
PC , we can write:
max z = cx
s.t. Ax ≤ b
x≥0
PS : max (min) z = cx
s.t. Ax = b
x ≥ 0.
3
ative), then we can replace xj everywhere by the difference between two new
00
variables x0j and xj where the latter are required to be nonnegative. A useful
exercise is to work through a few conversions between these two forms.
2.2 Modeling
As indicated earlier, the basic trick in operations research is to grapple with
a physical problem setting or system description and create ultimately, a
formulation or mathematical model that captures the requirements of the
real setting. Especially helpful, is to create a model that qualifies as a linear
program for as we will see subsequently, this constitutes a major step in the
direction of ultimately solving the problem. Following, we go through some
easy illustrations of the model-building activity.
Example 1: An oil refinery can buy two types of oil: light crude and heavy
crude. The cost per barrel is 11 and 9 dollars respectively. The portion of
gasoline, kerosene, and jet fuel produced per barrel of each type of crude is
given below:
Observe that the proportions do not add to 1.00 because there is some loss in
the refining process itself, i.e., 5 % in the case of light crude oil and 8 % for
heavy. Now, the refinery must deliver 1,000,000 barrels of gasoline, 400,000
barrels of kerosene, and 250,000 barrels of jet fuel. How can they meet this
production requirement at least total cost?
4
and jet fuel and we know how much of each, relatively speaking, we can
extract from each type of crude through the refining process. For example,
if we consider the production of gasoline, then we know that whatever our
amount of light and heavy crude oil, the amount of gasoline possible upon
refinement will be 0.4xL + 0.32xH and this total must be at least 1,000,000
barrels, i.e., 0.4xL + 0.32xH ≥ 1, 000, 000. The other constraints (relative to
kerosene and jet fuel) are handled in the same fashion. Finally, it makes no
sense to purchase a negative amount of crude oil which suggests nonnegativ-
ity restrictions on the variables and we are thus led to an LP model of the
stated problem:
xL , xH ≥ 0.
2
Example 2: A company in South America processes beans into coffee at its
m different production facilities. The coffee is then shipped to n warehouses
for retail distribution and exportation. A unit shipping cost (i.e., cost per
ton) from plant i to warehouse j is specified by cij . The production capacity
at plant i is ai and the demand called for at warehouse j is denoted by bj .
What is the least total cost production-shipping strategy for the company in
order that production capacity not be exceeded while satisfying the demands
at all of the warehouses?
5
any source, needs to be at least bj . Finally, the total cost of a shipping de-
cision xij is obviously cij xij . Combining all of this produces the following
formulation:
2
Example 3: Find the largest integer in the list a1 , a2 , . . . , an .
6
min z
s.t. z ≥ a1
z ≥ a2
..
.
z ≥ an .
2
In fact, there is little “science” involved in modeling optimization prob-
lems, including linear programming formulations; there are no algorithms
that specify a precise step-by-step receipt. That said, one usually gains
much from experience; very often apparently diverse problem settings in the
real world can be modeled correctly by exploiting similar tricks that have
proven successful in other recognizable settings. Indeed, effective modeling
is a bit of an art. Still, a reasonable general tactic to pursue when beginning,
is to reflect on what the problem is asking. In doing this, one usually is able
to specify the variables in the problem. Next, ask what it is in the problem
that restricts or constrains values that are assignable to the variables. In
progressing through this exercise, one is often very close to being able to
quantify the functional contraints in the formulation. To reiterate, there is
no magic key to formulating what can be very complicated story problems
but by progressing deliberately through the execise indicated, beginning stu-
dents are often surprised at how close they can get to creating complete and
valid formulations.
7
3.1 Geometry and Basic Notions
Let us consider a simple LP instance:
x1 , x2 ≥ 0. (3), (4)
Note that we have numbered the constraints for purposes of subsequent ref-
erence. Now, since our problem possesses only two variables, we are afforded
the luxury of creating a graphical depiction. To this end, consider Figure
1. Obviously, the nonnegativity constraints ((3) and (4)) require that our
search be restricted to the first orthant of the 2-dimensional space indicated.
Similarly, we can plot the other two constraints given by inequalities (1) and
(2) as shown. Obviously, if there is any nonempty space contained in the in-
tersection of the half-spaces described by these four inequalities then we have
at least one feasible solution for our problem. This solution set is referred
to as the feasible region; in Figure 1, it it is indicated by the shaded area.
The good news is that for the example used, we have a nonempty feasible
solution set. The bad news is that there are infinitely many feasible solutions
present; how then do we locate an optimal one?
Here’s an idea. Suppose we examine the objective function: z = 3x1 +2x2 .
This is just a line with slope - 32 and intercept z2 . Now, for some fixed value
of z, there are in infinite number of possibilities for x1 and x2 , i.e., every
point on the line. So if we vary values for z, this has the effect, graphically
speaking, of simply moving the line parallel to itself. Decreasing the value for
z moves the line towards the origin; larger values move it away. Since large
values for z are good (we are maximizing), we would want z to be as large as
possible so long as there were values for x1 and x2 that remained feasible, i.e.,
that intersected with the feasible region. But this means that we need only
shift (through the setting for z) the objective function line in the direction
of the gradient 5z = (3, 2), continuing until we reach the edge of the feasible
region. Since we have assumed all functions to be linear, there will be no
8
“surprises” in proclaiming the last point or points that are touched to be
optimal. In Figure 1, this movement is indicated by the dotted lines and
accordingly, the last “point of contact” with the feasible region is the single
point c corresponding to the values x1 = 32 and x2 = 1. We thus claim this is
the unique optimal solution; its value is z = 6.5. It is perhaps instructive to
exhibit the 3-dimensional representation of this sample problem by including
the axis corresponding to z. This is shown in Figure 2.
Now, our claim is that the simple approach made graphical by this ele-
mentary illustration, is enough, i.e., that it is extendable to the solution of
general linear programs. That is, we will simply pass the objective function
plane (i.e., hyperplane in arbitrary dimensions) through the feasible region
generated by the problem constraints, and observe the last admissible point
(or points) that intersect(s) with the plane accordingly. Of course, this is not
something that can be done graphically but if we can make the process alge-
braic then we might (if our claim is true) be close to producing an algorithm
for solving LPs.
9
3.1.1 The Feasible Region of Linear Programs
Consider the LP in standard form again; in fact, let us state it in the follow-
ing, rather formal way:
max(min) {cx | x ∈ S}
10
must be that x3 ≥ 0 since λ is nonegative and bounded by 1 and hence we
have arrived at the desired outcome that x3 ∈ S and thus the set S is convex.
Of course, we have assumed that the set S is nonempty and as we will see
later, for cases where in fact, there is no feasible solution to a problem, i.e.,
where S = ∅, we will need an unambiguous way to decide this.
11
Property 2: If a finite optima exists for an LP, then there is an optimal
extreme point.
2
Of course, there may be optimal solutions to an LP that are not extreme
points, but Property 2 indicates that if this is the case, then there has to also
be an optimal solution that does, in fact, correspond to an extreme point.
So then, in searching for an optimal solution, it is the case that we need only
search among the extreme points of feasible region. Of course, there could
exist more than one optimal extreme point but in this case, we know that
our LP would have an entire family of optimal solutions:
12
been parallel to say constraint 2, then the last part of the feasible region
shown that would have been touched by the objective function would be the
line segment between and including points b and c. Any of the infinitely
many points on this line segment would have been optimal.
2x1 + 3x2 + S1 = 6
4x1 + 2x2 + S2 = 8
x1 , x2 , S1 , S2 ≥ 0.
Observe that slack variables for each constraint are denoted by S1 and S2
respectively. Now, relative to the matrix representation of this standard form
construction, the matrix A appears as
13
x1 x2 S1 S2
!
2 3 1 0
A= .
4 2 0 1
We have indicated, across the top of the array, the variables identified with
the corresponding columns. The!vector representing the right-hand sides of
6
the system is given by b = . Now, suppose we consider the submatrix
8
of A formed by the columns corresponding to variables x1 and x2 . Calling
this B1 we have:
x1 x2
!
2 3
B1 = .
4 2
The basic variables are given by xB = (x1 , x2 ) which then implies that the
nonbasic variables (all of the others) result as xN = (S1 , S2 ). By definition
then, we have S1 = S2 = 0. Now, matrix B1 is invertible and we obtain
accordingly:
!
− 14 3
B−1 = 8
1 1
2
− 14
!
3
whereupon we can solve for xB as B−1
1 b = . That is, x1 = 32 and x2 =
2
1
1. Since all of the variables (including the slack variables) are nonnegative,
we have produced a basic feasible solution. Suppose we repeat this process
by forming another submatrix say B2 defined by the variables x1 and S1 , i.e.,
x1 S2
!
2 1
B2 = .
4 0
14
Of course, what we are trying to do is solve the system given by
2x1 + S1 = 6
4x1 = 8.
Basis xB (x1 , x2 , S1 , S2 )
B5 (x2 , S2 ) (0, 2, 0, 4)
B6 (S1 , S2 ) (0, 0, 6, 8)
The first two of these (given by B3 and B4 ) are not basic feasible solutions
since at least one of the variables takes on a negative value. Of course some
readers may wonder why this is relevant if the offending variables are only
slack variables. The answer is easy and is evident by simply examining
what the corresponding solution represents geometrically. Indeed, in Figure
5, we indicate precisely what each basis Bi corresponds to in the graphical
15
depiction of this sample problem instance. In the cases of B3 and B4 , what
we have actually located are solutions that correspond to the intersection
of certain problem constraints but unfortunately, not ones that define an
extreme point of the problem’s feasible region.
16
2
In fact, the establishment of this “equivalence” property is one of the fun-
damental underpinnings in the theory of linear programming; the previous
properties that were stated actually derive as corollaries of Property 4.
Since we saw that basic solutions were determined by forming invertible
m × m submatrices of the constraint matrix A, it follows that the total
number of basic!feasible solutions for an n-variable problem is bounded from
n
above by . Observe that n and m are defined by the LP stated in
m
standard form. In addition, the implied inequality follows since many candi-
date m × m submatrices will not contain linearly independent columns and
in addition, many that do, will not produce basic feasible solutions.
So, in theory we could solve an LP by simply examining its finitely many
basic feasible solutions and then pick the best. Unfortunately, such a strategy
would be tedious at best since for even small instances (e.g., n = 100, m =
25), this enumeration would be very time-consuming. But beyond this, there
are other, not so subtle, matters with which to be concerned. What if an
LP has no solution space? What if its solution space is not bounded? What
about multiple solutions? And what are those degenerate solutions that were
defined previously; can they be an issue? Clearly, we need some formal way
of addressing these matters.
17
during the second world war, work leading to the ultimate development of
the simplex algorithm stands as one of the celebrated success stories in op-
erations research (George Dantzig is often referred to as the father of linear
programming).
In concept, the simplex algorithm is essentially a bookkeeping strategy
for basic solution manipulation which, in turn, guides our search through
a sequence of adjacent extreme points of an LP’s feasible region. We need
three things:
• a stopping criterion
Rather than state the simplex algorithm in a formal way (which we will
do later), it should be instructive to motivate the basic notions called for
above. To this end, let us continue with the sample problem that has been
used thus far. For ease, the original (standard form) system is presented
again below:
z = 3x1 + 2x2
2x1 + 3x2 + S1 = 6
4x1 + 2x2 + S2 = 8
Observe that the slack variables take on coefficients of 0 and hence do not ap-
pear in the objective function equation. Although not stated as such, readers
should be clear that all variables are required to be nonnegative in the sys-
tem shown. Now, let us take as an initial basis the submatrix corresponding
to variables xB = (S1 , S2 ). For obvious reasons this is called the “all-slack”
basis. Certainly, this may not yield a very good initial solution but at least it
is feasible. Accordingly, let us solve for the slack variables in their respective
equations as follows:
S1 = 6 − 2x1 − 3x2 = 6
18
S2 = 8 − 4x1 − 2x2 = 8
.
Now, since x1 and x2 are nonbasic, they are set to 0 and we produce the
solution S1 = 6 and S2 = 8 which we (trivially) expected; its value is z = 0.
Obviously, the all slack solution corresponds to the origin (point a) in our
2-dimensional depiction of the feasible region.
Now, we observe that if the value of either x1 or x2 is elevated above 0,
we will improve upon the current (all slack) solution value. Since c1 = 3 >
2 = c2 , let us select variable x1 to make basic. Acting in a “greedy” fashion,
we would want to make x1 as large as is feasibly possible. But this is easy
to decide by simply evaluating the limit on how large x1 can be by examin-
ing the two current constraint equations above. From the first of these, we
see that x1 can be no greater than 3 while from the second equation, x1 is
bounded from above by 2. The smaller of these values defines the maximum
value that the variable can take on and still satisfy all contraints. So, let our
new basis consist of variables x1 and S1 , i.e., variable x1 has replaced slack
variable S2 from the previous basis. Solving for x1 in the second equation
yields:
x1 = 2 − x2
2
− S2
4
,
2x2 − S2
2
+ S1 = 2.
Obviously since variables x2 and S2 are now nonbasic and hence take on value
0, the new basic variables x1 and S1 both solve with value 2 accordingly. Ge-
ometrically, this means that we have moved from the basis associated with
the origin along the x1 axis, stopping at the extreme point b. Finally, if we
also substitute for x1 into our objective function, we obtain:
z =6+ x2
2
− 34 S2
19
z =6+ x2
2
− 34 S2
2x2 − S2
2
+ S1 = 2
x2 S2
x1 + 2
+ 4
= 2.
This new system is identical to the original one since all that has occurred
is a manipulation of equations, i.e., solutions feasible for one system are
preserved in the creation of the new system.
Now, in lookng at this new extreme point and the algebraic representation
given above, we can see that the corresponding basis can be improved since a
current nonbasic variable (x2 ) possesses a positive coefficient in the updated
objective function; the modified value of c2 is now 12 .
So, our intent is to bring into the basis, the improving variable x2 . Note
that had more than one nonbasic variable had a positive coefficient, we could
have used the previous rule-of-thumb selection policy of picking the variable
having the greatest such value. In any event, and owing to linearity, we
would seek to make x2 as large as possible and this means examining both
of the current constraint equations. Accordingly, from the first, we see that
the value of x2 can be no larger than 1 and from the second, no larger than
4. We are again bound by the smaller of these quantities and this results
from the solution in the first equation. That is, x2 will replace S1 from the
previous basis and solving as before yields:
x2 = 1 + S2
4
− S1
2
.
Substituing this expression into the current objective and second constraint
equations produces the new, equivalent system shown below:
z = 6 12 − S1
4
− 58 S2
x2 + S1
2
− S2
4
=1
x1 − S1
4
+ 38 S2 = 32 .
20
and hence, assignment of value 0, that we have produced the basic feasible
solution x1 = 32 and x2 = 1 with z = 6 12 .
This new basis corresponds to extreme point c. Moreover, the current
solution is optimal since we need only examine our equivalent system where-
upon we see that the modified objective row coefficients for nonbasic variables
(the only ones that could improve our current solution value) are nonposi-
tive, i.e., their introduction into a basis at any feasible, positive value could
only reduce the objective function value in hand. Therefore, we are prepared
to conclude that our sequence of moves from the initial extreme point (the
origin in this case) to the current one can stop with a claim that we have
found an optimal solution.
Now, the simplex algorithm (stated crudely of course) simply performs
the calculations just demonstrated in an organized way making use of tabular
representations. Hence, the initial simplex tableau would appear as follows:
x1 x2 S1 S2
−3 − 0 0 0
S1 2 3 1 0 6.
S2 4 2 0 1 8
The columns of the tableau are labeled by the variables; these labels never
change. The rows relate to the objective function (treated as an equation
beginning as z - cx = 0) and the constraints. The constraint rows are labeled
by the current basic variables, one per equation. As the initial tableau above
indicates, the starting basis consists of slack variables S1 and S2 , i.e., in any
basic tableau, the columns corresponding to the basic variables will be unit
vectors.
Now, we saw earlier that an improvement of our current (initial) solution
could be achieved by bringing into our basis the variable x1 as a replacement
for S2 . We solved for x1 in the second constraint equation and substituted
the outcome into the other constraint and the objective function. But this is
equivalent to performing elementary row operations on the original system
(first tableau) whereby a unit vector is created with the value 1 in the tableau
cell corresponding to the intersection of the entering and departing variables
respectively. This cell is referred to as the pivot cell of the tableau and the
21
algebraic manipulation is referred to as pivoting. If we do this, we obtain the
second tableau shown below:
x1 x2 S1 S2
0 −1/2 0 3/4 6
S1 0 2 1 −1/2 2
.
x1 1 1/2 0 1/4 2
The anxious reader may want to examine this tableau and compare the in-
formation captured accordingly, with that derived in the previous system of
equations after the first basis exchange. Obviously, the value of this basic
feasible solution is 6; the value is always read in the upper right-hand cell of
each tableau.
Upon examining the second tableau, the analogous test of optimality rel-
ative to potential improvement of our current objective function value would
now have us examining the objective function row and in particular, those
(updated) row coefficients of nonbasic variables to see if any existed at a
negative value. If so, then bringing one of these into a basis and throwing
something out would possibly produce another basis and if so, one that was
strictly better than our prevous one. Do not forget: looking for negative
objective coefficients here corresponds to our search for positive ones earlier
since in our tabular format, we have rewritten the objective function. In any
event, we observe that the coefficient of nonbasic x2 is -1/2 and so we will
bring it into the basis. The decision of which variable it will replace, is deter-
mined by a ratio test that, as we observed, simply allows us to make the new
variable as large as possible while satisfying the problem constraints. This
is a check that asks only that we examine the ratio of a tableau’s current
right-hand-side value to the entering variable row coefficient and pick the
smallest (negative values are not considered). In this case, our comparison
is between the ratios 22 and 1/22
suggesting that x2 will replace S1 in the first
constraint equation. We pivot on the relevant cell of the last tableau creating
the third one as displayed below:
22
x1 x2 S1 S2
0 0 1/4 5/8 6 12
x2 0 1 1/2 −1/4 1 .
x1 1 0 −1/4 3/8 3/2
All of the coefficients in the modified objective function row are now nonnega-
tive which is our stopping signal. The solution in this last tableau corresponds
to the basis consisting of x1 and x2 with values 3/2 and 1 respectively which
is consistent with our expectation given both our graphical and algebraic
solutions provided earlier. Again, skeptical readers might derive some com-
fort in examining the final tableau above and verifying for themselves that
the coefficients in the tableau are exactly those in the final set of equations
produced previously. Indeed, any interim simplex tableau simply reflects
the system of equations that would have been derived had our manipulation
proceeded in the routine fashion demonstrated initially.....no more, no less.
Of course, we are not ready to claim, in a precise sense, that we can solve
all linear programs. To be sure, there are a host of issues that have been left
rather vague to this point, i.e., how to determine if an LP has no solution,
has no finite solution value, etc.. Naturally, we will look at how one responds
to these issues; however, we have at least exposed the key notions that will
provide the underpinning for the resolution of any linear program and prior
to proceeding, we provide a high level statement of the fundamental process.
23
all i for which aij ∗ > 0. Pivot on cell (i∗ j ∗ ) and return to Step 1.
2
We emphasize again that the statement above, while sufficient to expose
the basic computation of the simplex algorithm, leaves rather substantial
gaps in how one might deal with what are practical, indeed very real-world,
requirements for negotiating general linear programming problems. Among
these are the following questions:
• What if there is no departing variable, i.e., what if the Step 2 test is not
satisfied?
• How can we detect when a problem has no finite optima; when it has mul-
tiple optima?
4 Generalizations
Our approach will continue to be an informal one, relying largely on illustra-
tions. In this regard, readers are advised to create their own instances and
to replicate the phenomena captured in the following examples. Accordingly,
it will be instructive to employ the graphical context afforded by small in-
stances in order to fully appreciate the relationships between the algebraic
and geometric interpretations of the various outcomes. First, we have to add
a final piece to our basic methodology: the notion of artificial variables.
24
4.1 Artificial Variables
Suppose we were given the problem instance below:
s.t. x1 + x2 + x3 = 7
2x1 − 5x2 + x3 ≥ 10
x1 , x2 , x3 ≥ 0.
x1 + x2 + x3 = 7
2x1 − 5x2 + x3 − S1 = 10
x1 , x2 , x3 , S1 ≥ 0.
25
to create a quick, albeit synthetic, starting basis without the overhead of any
computational effort. Denoting these variables as R1 and R2 , and requiring
these to also be nonnegative, the new system of constraints becomes:
s.t. x1 + x2 + x3 + R1 = 7
2x1 − 5x2 + x3 − S1 + R2 = 10
x1 , x2 , x3 , S1 , R1 , R2 ≥ 0.
x1 x2 x3 S1 R1 R2
!
1 1 1 0 1 0
A=
2 −5 1 −1 0 1
.
Then, the scheme is to simply apply the simplex algorithm and hope that
upon its application, we will ultimately eliminate the artificial variables since
they are, by definition, not really part of the true formulation. There are two
well-known approaches for dealing with artificial variables:
• big-M method;
• two-phase method.
4.1.1 Big-M
The big-M method of handling instances with artificial variables is the “common-
sense” approach. Essentially, the notion is to make the artificial variables,
through their coefficients in the objective function, so costly or unprofitable
that any feasible solution to the real problem would be preferred....unless
the original instance possessed no feasible solutions at all. But this means
26
that we need to assign, in the objective function, coefficients to the artificial
variables that are either very small (maximization problem) or very large
(minimization problem); whatever this value, let us call it big M. In fact,
this notion is an old trick in optimization in general; we simply associate a
penalty value with variables that we do not want to be part of an ultimate
solution (unless such an outcome is unavoidable). Indeed, the penalty is so
costly that unless any of the respective variables’ inclusion are warranted
algorithmically, such variables will never be part of any feasible solution.
So, the objective function for this example, upon the addition of the ar-
tificial variables, would appear as follows:
Obviously, any nonzero (recall that all variables are required to be nonneg-
ative) value for R1 or R2 would occur in a final solution if and only if there
were no alternatives involving the other, real variables.
Now, placing the formulation in tableau form, and employing only artifi-
cial variables in the starting basis, we have:
x1 x2 x3 S1 R1 R2
−2 −3 5 0 M M 0
R1 1 1 1 0 1 0 7 .
R2 2 −5 1 −1 0 1 10
Making this tableau “basic” produces (remember that we need unit vectors
under the R1 and R2 columns):
x1 x2 x3 S1 R1 R2
−2 − 3M −3 + 4M 5 − 2M M 0 0 −17M
R1 1 1 1 0 1 0 7 .
R2 2 −5 1 −1 0 1 10
27
At this point we know what to do. Applying the simplex algorithm, we
can bring into the basis, variable x1 and drive out (per the ratio check) arti-
ficial variable R2 yielding:
x1 x2 x3 S1 R1 R2
0 −8 − 72 M 6 − 12 M −1 − 12 M 0 1 + 32 M 10 − 2M
R1 0 7
2
1
2
1
2
1 − 12 2 .
x1 1 − 52 1
2
− 12 0 1
2
5
x1 x2 x3 S1 R1 R2
0 0 50
7
1
7
16
7
+M − 17 +M 102
7
x2
0 1 1
7
1
7
2
7
− 17 4
7
.
x1 1 0 6
7
− 17 5
7
1
7
45
7
Now, at this point there are no nonpositive coefficients in the objective row
of the current tableau (remember that M is a very large value) which is
our sign that we are done. The final and hence optimal solution is x1 = 45 7
and x2 = 47 . All other variables, by virtue of their being nonbasis, take on
value 0. Particularly meaningful (“nonbasic-ness” aside), is that the artificial
variables have now disappeared. As we intimated earlier, if we are to solve the
real problem, this disappearance is not just important; in fact, it is essential.
28
shall see in the next section, must possess no feasible solution space. For the
present, however, let us assume the former outcome. Then, we would remove
the artificial variable columns from the final tableau of phase-I and enter
phase-II with an admissible basis (the one produced at the end of phase-I).
We then reapply the simplex at this point and continue in the normal fashion.
An illustration should make the approach plain.
Consider the previous instance with the new objective function, consisting
of only artificial variables, as indicated. That is, our phase-I instance would
be:
min z = R1 + R2
s.t. x1 + x2 + x3 + R1 = 7
2x1 − 5x2 + x3 − S1 + R2 = 10
x1 , x2 , x3 , S1 , R1 , R2 ≥ 0.
Now, the initial tableau for phase-I (after having been made basic) ap-
pears as follows:
x1 x2 x3 S1 R1 R2
3 −4 2 −1 0 0 17
R1
1 1 1 0 1 0 7 .
R2 2 −5 1 −1 0 1 10
x1 x2 x3 S1 R1 R2
0 7
2
1
2
1
2
0 − 32 2
R1 0 7
2
1
2
1
2
1 − 12 2.
x1 1 − 52 1
2
− 12 0 1
2
5
29
Next, we pivot in x2 and R1 departs. We have:
x1 x2 x3 S1 R1 R2
0 0 0 0 −1 −1 0
x2
0 1 1
7
1
7
2
7
− 17 4
7
.
x1 1 0 6
7
− 17 5
7
1
7
45
7
But this is a final tableau (nonpositive objective row coefficients for minimiza-
tion) and so we have concluded phase-I. More importantly, we have driven
out the artificial variables (observe that the tableau objective function row
value of 0 is consistent with this outcome). So, we can proceed to phase-II
where operationally, we extract the relevant part of the above (phase-I final)
tableau and strip off the first row replacing it with the original objective
function, yielding the following:
x1 x2 x3 S1
−2 −3 5 0 0
x2
0 1 1
7
1
7
4
7
.
x1 1 0 6
7
− 17 45
7
x1 x2 x3 S1
50 1 102
0 0 7 7 7
x2
0 1 1
7
1
7
4
7
x1 1 0 6
7
− 17 45
7
which is the starting tableau for phase-II. Interestingly, however, this tableau
is optimal. Of course, readers should be clear that for real instances this
phenomenon will not typically occur; further pivoting will be required on
the initial phase-II tableau. In any event, it is instructive to examine the
phase-II tableau above and compare it with the corresponding “subtableau”
30
produced earlier by the big-M method; as we would expect, they are the
same.
Obviously, appending artificial variables to a problem is only a “trick”
that serves to allow the simplex calculations to at least commence. From
the simplex algorithm’s perspective, its application is routine; in the real
problem context, however, we are pivoting through artificial bases with the
intended effect of ultimately “hooking up” with real bases (extreme points
for the oiginal problem) if possible. Geometrically, the process is captured
in Figure 6.
max z = 2x1 + x2
31
s.t. x1 + x2 ≤ 3
2x1 + 3x2 ≥ 12
x1 , x2 ≥ 0.
s.t. x1 + x2 + S1 = 3
2x1 + 3x2 − S2 + R1 = 12
x1 , x2 , S1 , S2 , R1 ≥ 0.
Now, beginning with an initial basis that includes S1 and R1 , we would pro-
ceed until stopping occurs with the tableau below (the computation is left as
an exercise for the reader):
x1 x2 S1 S2 R1
−1 + M 0 1 + 3M M 0 3 − 3M
x2
1 1 1 0 0 3
.
R1 −1 0 −3 −1 1 3
32
order to see if the outcome of phase-I’s feasibility test is consistent with the
conclusion drawn above.
x1 + 2x2 ≤ 6
x1 , x2 ≥ 0.
33
objective function through this space, in the improving direction, it is evident
that the last point of contact is the line segment indicated in bold. Thus,
every point on this line segment is an optimal solution. But this occurs since
the objective function is parallel to the first constraint in the instance where
the latter is binding in the sense that it defines a portion of the feasible region
specifying an optimal extreme point (obviously, the second constraint in the
original instance is redundant).
But since we don’t solve real-world linear programs graphically, what then
is the “signal” vis-a-vis simplex computations, that would indicate multiple
optima? Following is the final tableau that results upon a correct application
of the simplex algorithm to this example:
x1 x2 S1 S2
1
0 0 2
0 6
x2 2
3 1 1
6
0 2.
S2 − 13 0 − 13 1 2
34
The basic variables are x2 and S2 which corresponds to the actual point
x1 = 0 and x2 = 2. But observe that there is an objective row coefficient
of 0 under the nonbasic variable x1 (in the final tableau) . This indicates
that there may be another optimal basis. Indeed, if we pivot in variable x1
(on this tableau) to replace x2 , another basis, consisting of variables x1 and
S2 , results with values of 3 and 0 respectively. The latter corresponds to
the extreme point given (graphically) by x1 = 3 and x2 = 0. But this is
to be expected because we just argued that the line segment between these
two extreme points of the feasible region (say x1 and x2 ) defined an entire
family of optimal solutions for the given instance, i.e., any point on this line
segment is optimal. Put more formally, every solution x0 = λx1 + (1 − λ)x2
for all 0 ≤ λ ≤ 1 is an optimal solution to the indicated instance. The reader
is invited to test this claim by picking any such λ value and checking if the
outcome for x0 yields a value for z of 6. Of course, for any λ ∈ (0, 1), the
resulting x0 is a nonbasic solution.
s.t. x1 − 2x2 ≤ 10
2x1 − x2 ≤ 10
x1 , x2 ≥ 0.
35
Starting with the all-slack basis, we would ultimately reach the following
tableau:
x1 x2 S1 S2
0 − 12 0 3
2
15
S1 0 − 32 1 − 12 5
.
x1 1 − 12 0 1
2
5
36
4.5 Degeneracy
Our final property relates to degeneracy. Recall that a degenerate basis is one
in which at least one basic variable has value 0. Essentially, degeneracy is a
“non-issue” in solving practical linear programs, but only if certain safeguards
are in place; absent such safeguards, degeneracy can be a substantial problem.
Consider the following instance:
max z = 2x1 + x2
s.t. x1 + x2 ≤ 10
x1 + 3x2 ≤ 10
x1 , x2 ≥ 0.
Starting with the all-slack initial basis, the first tableau appears as follows:
x1 x2 S1 S2
−2 −1 0 0 0
S1 1 1 1 0 10
.
S2 1 3 0 1 10
x1 x2 S1 S2
0 1 2 0 20
x1 1 1 1 0 10
.
S2 0 2 −1 1 0
This is a final tableau; the optimal solution is given by the final basis con-
sisting of x1 and S2 accordingly. But the final basis is degenerate since
a basic variable exists at a value of 0. What does this mean graphically?
Essentially, what a degenerate solution implies is that an extreme point is
37
“over-specified.” That is, the extreme point is represented by more than one
basis. Figure 10 helps to make the point clear. Here, the final extreme point
is represented by the intersection of three constraints when two would suffice.
x2 + 12 x4 − 12x5 − 12 x6 + 3x7 = 0
x3 + x6 = 1
x1 , x2 , x3 , x4 , x5 , x6 , x7 ≥ 0.
38
produce the following succession of bases (after B1 ):
B2 = (x4 , x2 , x3 ) with z = 0
B3 = (x4 , x5 , x3 ) with z = 0
B4 = (x6 , x5 , x3 ) with z = 0
B5 = (x6 , x7 , x3 ) with z = 0
B6 = (x1 , x7 , x3 ) with z = 0
B7 = (x1 , x2 , x3 ) with z = 0
5 Final Comments
In concluding this basic coverage of linear programming, it is worth remarking
that what we have presented here is but an introduction to what amounts
to a most powerful and exceptionally rich optimization procedure. As we
indicated at the outset, the methodology of linear programming represents a
very important tool in the problem-solving arsenal of engineers and applied
mathematicians who are charged with dealing with critical problems that
arise on a daily basis in the world of business and industry.
39
Still, much of the richness, indeed, much of what is to be appreciated
in linear programming must remain beyond the scope of this presentation.
For example, the entire theory of linear programming duality represents an
elegant development in its own right but more importantly, provides the
basis for much of what constitutes the real power in the application of linear
programming to real-world problems. The related concepts of post-optimality
and sensitivity analysis are enormously powerful notions that derive from
duality theory.
In addition, we do not have space to take up the cases of so-called spe-
cial LP structures. These are linear programs that exhibit characteristics
in their model formulations that allow shortcuts or stream-lined approaches.
Interestingly, the arguments that are used to justify the correctness of these
shortcuts also often stem from notions in duality. In any event, readers in-
terested in these and other topics are directed to the various sources in the
reference list where a number of excellent, full treatises of linear programming
are cited.
40
6 Exercises
1. Place each of the formulations below in standard form.
(a)
min z = 4x1 + 2x2 − 33x3
s.t. x1 − 4x2 + x3 ≤ 12
9x1 + 6x3 = 15
−5x1 + 9x2 ≥ 3
x1 , x2 , x3 ≥ 0.
(b)
max z = 45x1 + 15x3
−2x1 + 5x2 − x3 ≥ 1
x1 − x2 ≤ 3
x1 , x2 , x3 ≥ 0.
(c)
min z = 2x1 + x2 − 4x3
s.t. x1 − x2 − 5x3 ≤ 10
3x2 + 9x1 = −6
x1 ≥ 0, x3 ≤ 0, x2 unrestricted in sign.
41
(d)
min z = 3x1 − 3x2 + 7x3
s.t. x1 + x2 + 3x3 ≤ 40
x1 + 9x2 − 7x3 ≥ 50
| 5x2 + 8x3 |≤ 70
x1 , x2 ≥ 0, x3 unrestricted in sign.
s.t. x1 + x2 ≤ 4
4x1 + 3x2 ≤ 12
−x1 + x2 ≥ 1
x1 , x2 ≥ 0.
3. Solve graphically:
s.t. 0 ≤ x1 ≤ 5.
42
4. Consider the following instance:
x1 , x2 , x3 , x4 ≥ 0.
Determine:
3x1 − x2 + x3 + 2x4 ≤ 8
2x1 + 3x2 − x3 + x4 ≤ 10
x1 , x2 , x3 , x4 ≥ 0.
43
6. Solve the following minimization problem using the simplex algorithm.
−2x1 + 4x2 ≤ 12
x1 , x2 , x3 ≥ 0.
7. Solve the following instance with the simplex; start with variables x4 , x5 ,
and x6 in the initial basis.
3x1 − x6 = 0
x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.
8. Solve the following problem by employing both the big-M and the two-
phase methods.
s.t. x1 + x2 + x + 3 = 7
2x1 − 5x2 + x3 ≥ 10
x1 , x2 , x3 ≥ 0.
44
9. Find all of the alternative optimal basic solutions to the following instance
and then write a general expression for all of the nonbasic solutions.
x1 + x2 ≤ 5
x1 ≤ 1
x1 , x2 , x3 ≥ 0.
10. Solve the following linear program by inspection and then justify the
outcome in terms of the simplex algorithm.
x1 , x2 , x3 , x4 , x5 ≥ 0.
11. Solve the following problem, showing that the optimal solution is degen-
erate and that there exist alternative solutions that are all nonbasic.
max z = 3x1 + x2
s.t. x1 + 2x2 ≤ 5
x1 + x2 − x3 ≤ 2
x1 , x2 , x3 ≥ 0.
12. In which direction is the solution space for the problem instance below
unbounded?
45
max z = 20x1 + 10x2 + x3
x1 + x3 ≤ 10
x1 − x2 + 4x3 ≤ 20
x1 , x2 , x3 ≥ 0.
13. Apply the big-M method to the formulation below and conclude, accord-
ingly, that there are no feasible solutions.
s.t. 2x1 + x2 + x3 ≤ 2
x1 , x2 , x3 ≥ 0.
14. Repeat exercise 13 using the two-phase method for artificial variables.
15. Suppose that at the end of phase-I, there is an artificial variable in the
basis at a value of 0. Is this a problem? What would you do?
46
7 References
There have been many books written on the subject of linear programming.
These range from ones at an exceptionally high level (i.e., research mono-
graphs), to others that are quite basic and elementary. Following, we give a
very short list of references that should be instructive to students pursuing
technical degrees in fields such as engineering, mathematics, computer sci-
ence and the like. Ones marked by bold reference numbers tend to be more
advanced.
47