Any variable not already constrained to be nonnegative is replaced by the difference of two new variables which are so
constrained.
Linear constraints are of the form:
i
n
j
j ij
b x a ~
1
where ~ stands for one of the relations , , = not necessarily the same one for each i!. "he constants bi may always be ass#med
nonnegative.
$%ample: "he constraint 2x1 3x2 + 4x3 5 is m#ltiplied by 1 to obtain
2x1 + 3x2 4x3 5, which has a nonnegative right&hand side.
Slack Variables and Surplus Variables
A linear constraint of the form
i j ij
b x a
can be converted into an e'#ality by adding a new, nonnegative variable to the
left&hand side of the ine'#ality. (#ch a variable is n#merically e'#al to the difference between the right& and left&hand sides of the
ine'#ality and is )nown as a slack variable.
Example: A man#fact#rer is beginning the last wee) of prod#ction of fo#r different models of wooden television consoles,
labeled *, **, ***, and *+, each of which m#st be assembled and then decorated. "he models re'#ire ,, , ., and  h, respectively,
for assembling and /, 1., ., and . h, respectively, for decorating. "he profits on the models are 01, 0,, 02, and 03, respectively.
"he man#fact#rer has .4 444 h available for assembling these prod#cts 14 assemblers wor)ing ,4 h5w)! and /4 444 h available
for decorating 44 decorators wor)ing ,4 h5w)!. 6ow many of each model sho#ld the man#fact#rer prod#ce d#ring this last
wee) to ma%imi7e profit8 Ass#me that all #nits made can be sold.
"he first constraint in above problem is
4x1 + 5x2 + 3x3 + 5x4 30 000
"his ine'#ality is transformed into the e'#ation
4x1 + 5x2 + 3x3 + 5x4 9 x5 = 30 000
by adding the slac) variable x5 to the left&hand side of the ine'#ality. 6ere x5 represents the n#mber of assembly ho#rs available to
the man#fact#rer b#t not #sed.
A linear constraint of the form
i j ij
b x a
can be converted into an e'#ality by s#btracting a new, nonnegative variable from
the left&hand side of the ine'#ality. (#ch a variable is n#merically e'#al to the difference between the left& and right&hand sides of
the ine'#ality and is )now as a s#rpl#s variable. *t represents e%cess inp#t into that phase of the system modeled by the constraint.
Example: :niversal ;ines *nc. operates three mines in <est +irginia. "he ore from each mine is separated into two grades
before it is shipped= the daily prod#ction capacities of the mines, as well as their daily operating costs, are as follows:
6igh&>rade ?re,
tons5day
Low&>rade ?re,
tons5day
?perating @ost, 014445
day
;ine * , , /4
;ine ** 2 , //
;ine *** 1 2 1A
:niversal has committed itself to deliver , tons of high&grade ore and 2 tons of low&grade ore by the end of the wee). *t also has
labor contracts that g#arantee employees in each mine a f#ll dayBs pay for each day or fraction of a day the mine is open.
Cetermine the n#mber of days each mine sho#ld be operated d#ring the #pcoming wee) if :niversal ;ines is to f#lfill its
commitment at minim#m total cost.
"he first constraint in the above problem is
4x1 + 6x2 + x3 54
"he left&hand side of this ine'#ality represents the combined o#tp#t of high&grade ore from three mines, while the right&hand side
is the minim#m tonnage of s#ch ore re'#ired to meet contract#al obligations. "his ine'#ality is transformed into the e'#ation
4x1 + 6x2 + x3 x4 = 54
by s#btracting the s#rpl#s variable x4 from the left&hand side of the ine'#ality. 6ere x4 represents the amo#nt of high&grade ore
mined over and above that needed to f#lfill the contract.
1&4
Generating an Initial Feasible Solution
After all linear constraints with nonnegative right&hand sides! have been transformed into e'#alities by introd#cing slac) and
s#rpl#s variables where necessary, add a new variable, called an artificial variable, to the left&hand side of each constraint
e'#ation that does not contain a slac) variable. $ach constraint e'#ation will then contain either one slac) variable or one artificial
variable. A nonnegative initial sol#tion to this new set of constraints is obtained by setting each slac) variable and each artificial
variable e'#al to the right&hand side of the e'#ation in which it appears and setting all other variable, incl#ding the s#rpl#s
variable, e'#al to 7ero.
Example: "he set of constraints
x1 + 2x2 3
4x1 + 5x2 6
7x1 + 8x2 = 15
is transformed into a system of e'#ations by adding a slac) variable, x3, to the left&hand side of the first constraint and s#btracting
a s#rpl#s variable, x4, from the left&hand side of the second constraint. "he new system is
x1 + 2x2 + x3 = 3
4x1 + 5x2 x4 = 6
7x1 + 8x2 = 15
*f now artificial variables x5 and x6 are respectively added to the left&hand sides of the last two constraints in system, the
constraints witho#t a slac) variable, the res#lt is
x1 + 2x2 + x3 = 3
4x1 + 5x2 x4 + x5 = 6
7x1 + 8x2 + x6 = 15
A nonnegative sol#tion to this last system is x3 = 3, x5 = 6, x6 = 15, and x1 = x2 = x4 = 0. Dotice, however, that x1 = 0, x2 = 0 is
not a sol#tion to the original set of constraints.!
Penalty Costs
"he introd#ction of slac) and s#rpl#s variables alters neither the nat#re of the constraints nor the obEective. Accordingly, s#ch
variables are incorporated into the obEective f#nction with 7ero coefficients. Artificial variables, however, do change the nat#re of
the constraints. (ince they are added to only one side of an e'#ality, the new system is e'#ivalent to the old system of constraints
if and only if the artificial variables are 7ero. "o g#arantee s#ch assignments in the optimal sol#tion in contrast to the initial
sol#tion!, artificial variables are incorporated into the obEective f#nction with very large positive coefficients in a minimi7ation
program or very large negative coefficients in a ma%imi7ation program. "hese coefficients, denoted by either or , where
is #nderstood to be a large positive n#mber, represent the severe! penalty inc#rred in ma)ing a #nit assignment to the artificial
variables.
Standard atrix Form:
;inimi7e: ! = x1 + 2x2 + 3x3
(#bEect to: 3x1 + 4x3 5
5x1 + x2 + 6x3 = 7
8x1 + "x3 2
<ith: all variables nonnegative
Adding a slac) variable x4 to the left&hand side of the first constraint, s#btracting a s#rpl#s variable x5 from the left&hand side of
the third constraint, and then adding an artificial variable x6 only to the left&hand side of the third constraint, we obtain the
program
;inimi7e: ! = x1 + 2x2 + 3x3 + 0x4 + 0x5 + x6
(#bEect to: 3x1 + 4x3 + x4 = 5
5x1 + x2 + 6x3 = 7
8x1 + "x3 # x5 + x6 = 2
<ith: all variables nonnegative
"his program is in standard form, with an initial feasible sol#tions x4 = 5, x2 = 7, x6 = 2, x1 = x3 = x5 = 0.
define
$ F Gx1, x2, x3, x4, x5, x6H
%
& F G1, 2, 3, 0, 0, H
%
/&4
1
1
1
]
1
1 1 4 3 4 A
4 4 4 2 1 
4 4 1 , 4 .
'
1
1
1
]
1
/
1

(
1
1
1
]
1
2
/
,
4
x
x
x
$
*n this case, x2 can be #sed to generate the initial sol#tion rather than adding an artificial variable to the second constraint to
achieve the same res#lt. *n general, whenever a variable appears in one and only one constraint e'#ation, and there with a positive
coefficient, that variable can be #sed to generate part of the initial sol#tion by first dividing the constraint e'#ation by the positive
coefficient and then setting the variable e'#al to the right&hand side of the e'#ation= an artificial variable need not be added to the
e'#ation.
!"e Simplex !ableau
"he simple% method is a matri% proced#re for solving linear programs in the standard form
?ptimi7e: ! = &
%
$
(#bEect to: '$ = (
<ith: $ 0
<here ( 0 and a basic feasible sol#tion $0 is )nown. (tarting with $0, the method locates s#ccessively other basic feasible
sol#tions having better val#es of the obEective, #ntil the optimal sol#tion is obtained. Ior minimi7ation programs, the simple%
method #tili7es tablea#, in which &0 designates the cost vector associated with the variables in $0.
( & ' & &
( ' & $
&
$
% % %
%
%
4 4
4 4
Ior ma%imi7ation programs, tablea# applies if the elements of the bottom row have their signs reversed.
Example: Ior the minimi7ation program of above problem, &0 = G4, /, ;H
%
. "hen
[ ] [ ]
1
1
1
]
1
1 1 4 3 4 A
4 4 4 2 1 
4 4 1 , 4 .
, / , 4 , 4 , 4 , . , / , 1
4
' & &
% %
[ ] [ ] + + , 4 , 3 1/ , / , A 14 , 4 , 4 , . , / , 1
[ ] 4 , , 4 , 3 3 , 4 , A 3
[ ] ( &
%
/ 1,
/
1

, / , 4
4
1
1
1
]
1
And tablea# becomes
x
x
x
x x x x x x
/ 1, 4 4 3 3 4 A 3
/ 1 1 4 3 4 A
1 4 4 4 2 1  /
 4 4 1 , 4 . 4
4 4 . / 1
2
/
,
2  , . / 1
# !ableau Simpli$ication
Ior each j j = 1, /, J , n!, define !j =
%
&
4
'j, the dot prod#ct of &0 with the j th col#mn of '. "he j th entry in the last row of
"ablea# is cj !j or, for a ma%imi7ation program, !j cj!, where cj is the cost in the second row of the tablea#, immediately above
'j. ?nce this last row has been obtained, the second row and second col#mn of the tablea#, corresponding to &
%
and &0,
respectively, become s#perfl#o#s and may be eliminated.
!"e Simplex et"od
.&4
(tep 1: Locate the most negative n#mber in the bottom row of the simple% tablea#, e%cl#ding the last col#mn, and call the col#mn
in which this n#mber appears the wor) col#mn. *f more than one candidate for most negative n#mbers e%ists, choose one.
(tep /: Iorm ratios by dividing each positive n#mber in the wor) col#mn, e%cl#ding the last row, into the element in the same row
and last col#mn. Cesignate the element in the wor) col#mn that yields the smallest ration as the pivot element. *f more
that one element yields the same smallest ration, choose one. *f no element in the wor) col#mn is positive, the program
has no sol#tion.
(tep .: :se elementary row operations to convert the pivot element to 1 and then to red#ce all other elements in the wor) col#mn
to 4.
(tep ,: Keplace the x&variable in the pivot row and first col#mn by the x&variable in the first row and pivot col#mn. "his new first
col#mn is the c#rrent set of basic variables.
(tep : Kepeat (teps 1 thro#gh , #ntil there are no negative n#mbers in the last row, e%cl#ding the last col#mn.
(tep 2: "he optimal sol#tion is obtained by assigning to each variable in the first col#mn that val#e in the corresponding row and
last col#mn. All other variables are assigned the val#e 7ero. "he associated 7L, the optimal val#e of the obEective
f#nction, is the n#mber in the last row and last col#mn for a ma%imi7ation program, b#t the negative of this n#mber for a
minimi7ation program.
odi$ication $or Programs %it" #rti$icial Variables
<henever artificial variables are part of the initial sol#tion $4, the last row of "ablea# will contain the penalty cost . "o
minimi7e ro#ndoff error, the following modifications are incorporated into the simple% method= the res#lting algorithm is the two&
phase method.
@hange 1: "he last row of "ablea# is decomposed into two rows, the first of which involves those terms not containing , while
the second involves the coefficients of ; in the remaining terms.
Example: "he last row of the tablea# in the last e%ample is
/ 1, 4 , , 4 , 3 3 , 4 , A 3
:nder @hange 1 it wo#ld be transformed into the two rows
1, 4 4 4 3 4 3
/ 4 1 4 3 , 4 A
@hange /: (tep 1 of the simple% method is applied to the last row created in @hange 1 followed by (tep /, ., and ,!, #ntil this
row contains no negative elements. "he (tep 1 is applied to those elements in the ne%t&to&last row that are positioned
over 7eros in the last row.
@hange .: <henever an artificial variable ceases to be basic M i.e., is removed from the first col#mn of the tablea# as a res#lt of
(tep , M it is deleted from the top row of the tablea#, as is the entire col#mn #nder it. "his modification simplifies hand
calc#lations b#t is not implemented in many comp#ter programs.!
@hange ,: "he last row can be deleted from the tablea# whenever it contains all 7eros.
@hange : *f non7ero artificial variables are present in the final basic set, then the program has no sol#tion. *n contrast, 7ero&
val#ed artificial variables may appear as basic variables in the final sol#tion when one or more of the original constraint
e'#ations is red#ndant.!
Solved Problems
Problem &
;a%imi7e: ! = x1 + "x2 + x3
(#bEect to: x1 + 2x2 + 3x3 "
3x1 + 2x2 +2x3 15
<ith: all variables nonnegative
"his program is p#t into matri% standard form by first introd#cing slac) variables x4 and x5 in the first and second constraint
ine'#alities, and then defining
$ F Gx1, x2, x3, x4, x5H
%
& F G1, ",1, 0, 0H
%
1
]
1
1 4 / / .
4 1 . / 1
'
1
]
1
1
3
(
1
]
1

,
4
x
x
$
"he costs associated with the components of $0, the slac) variable, are 7ero= hence &0 F G4, 4H
"
.
"ablea# becomes
,&4
1 1 4 / / . 4
3 4 1 . / 1 4
4 4 1 3 1

,
 , . / 1
x
x
x x x x x
"o comp#te the last row of this tablea#, we #se the tablea# simplification and first calc#late each !j by inspection: it is the dot
prod#ct of col#mn / and the j th col#mn of '. <e then s#btract the corresponding cost cj from it ma%imi7ation program!. *n this
case, the second col#mn is 7ero, and so !j cj = 0 cj = cj. 6ence, the bottom row of the tablea#, e%cl#ding the last element, is
E#st the negative of row /. "he last element in the bottom row is simply the dot prod#ct of col#mn / and the final, (&col#mn, and
so it too is 7ero. At this point, the second row and second col#mn of the tablea# are s#perfl#o#s. $liminating them, we obtain
tablea# as the complete initial tablea#.
4 4 4 1 3 1 !
1 1 4 / / .
3 4 1 . / 1
4 4 1 3 1

L
,
 , . / 1
j j
c !
x
x
x x x x x
<e are now ready to apply the simple% method. "he most negative element in the last row of tablea# is 3, corresponding to the
x2&col#mn= hence this col#mn becomes the wor) col#mn. Iorming the ratios 35/ = ,. and 15/ = 1., we find that the element /,
mar)ed b the asteris) in the tablea#, is the pivot element, since it yields the smallest ratio. "hen, applying (teps . and , to the
tablea#, we obtain ne%t tablea#. (ince the last row of the last table contains no negative elements, it follows from (tep 2 that the
optimal sol#tion is x2 = ")2, x5 = 6, x1 = x3 = x4 = 0, with ! = 81)2.
Problem '
;inimi7e: ! = 80x1 + 60x2
(#bEect to: 0*20x1 + 0*32x2 0*25
x1 + x2 = 1
with: x1 and x2 nonnegative
Adding a slac) variable x3 and an artificial variable x4 to the first and second constraints, respectively, we convert the program to
standard matri% form, with
$ F Gx1, x2, x3, x4H
%
& F G80,60,0, H
%
1
]
1
1 4 1 1
4 1 ./ . 4 /4 . 4
'
1
]
1
1
/ . 4
(
1
]
1
,
.
4
x
x
$
(#bstit#ting these matrices, along with &0 F G4, ;H
"
, we obtain initial tablea#.
x
x
x x x x
4 4 24 A4
1 1 4 1 1
/ . 4 4 1 ./ . 4 /4 . 4 4
4 24 A4

.
, . / 1
(ince the bottom row involves ;, we apply @hange 1= the res#lting tablea# is the initial tablea# for the two&phase method.
1 4 4 1 1
4 4 4 24 A4 !
1 1 4 1 1
/ . 4 4 1 ./ . 4 /4 . 4
L
,
.
, . / 1
j j
! c
x
x
x x x x
:sing both (tep 1 of the (imple% method and @hange /, we find that the most negative element in the last row of tablea#
e%cl#ding the last col#mn! is M 1, which appears twice. Arbitrarily selecting the x1&col#mn as the wor) col#mn, we form the
&4
ratios 4./54./4 = 1./ and 151 = 1. (ince the element 1, starred in tablea#, yields the smallest ratio, it becomes the pivot. "hen,
applying (tep . and , and @hange . to tablea# we generate ne%t tablea#.
4 4 4 4
A4 4 /4 4
1 4 1 1
4 . 4 1 1/ . 4 4
1
L
.
. / 1
x
x
x x x
?bserve that x1 replaces the artificial variable x4 in the first col#mn of tablea#, so that the entire x4&col#mn is absent from the
tablea#. Dow, with no artificial variables in the first col#mn and with @hange . implemented, the last row of the tablea# sho#ld be
all 7eros. *t is= and by @hange , this row may be deleted, giving
A4 4 /4 4
as the new last row of tablea#.
Kepeating (teps 1 thro#gh ,, we find that the x2&col#mn is the new wor) col#mn, the starred element in the tablea# is the new
pivot, and the elementary row operations yield the ne%t tablea#, in which all calc#lations have been ro#nded to fo#r significant
fig#res. (ince the last row of the tablea#, e%cl#ding the last col#mn, contains no negative elements, it follows from (tep 2 that x1
= 0*5833, x2 = 0*4167, x3 = x4 = 0, +it, ! = 71*67
21 . 11 1 . 122 4 4
A.. . 4 ... . A 4 1
,121 . 4 ... . A 1 4
1
/
. / 1
x
x
x x x
Problems:
:se the simple% or two&phase method to solve the following problems.
..1
;a%imi7e: ! = x1 + x2
s.t. x1 + 5x2 5
2x1 + x2 4
<ith: x1, x2 nonnegative
..12
;a%imi7e: ! = 3x1 + 4x2
s.t. 2x1 + x2 6
2x1 + 3x2 "
<ith: x1, x2 nonnegative
..11
;inimi7e: ! = x1 + 2x2
s.t. x1 + 3x2 11
2x1 + x2 "
<ith: x1, x2 nonnegative
..1A
;a%imi7e: ! = Mx1 M x2
s.t. x1 + 3x2 5000
2x1 + x2 12000
<ith: x1, x2 nonnegative
..13
;a%imi7e: ! = 2x1 + 3x2 + 4x3
s.t. x1 + x2 + x3 1
x1 + x2 + 2x3 = 2
3x1 + 2x2 + x3 4
<ith: x1, x2, x3 nonnegative
!"e (ual Simplex et"od
"he reg#lar! simple% method moves the initial feasible b#t nonoptimal sol#tion to an optimal sol#tion while maintaining
feasibility thro#gh an iterative proced#re. ?n the other hand, the d#al simple% method moves the initial optimal b#t infeasible
sol#tion to a feasible sol#tion while maintaining optimality thro#gh an iterative proced#re.
Iterative procedure o$ t"e (ual Simplex et"od:
(tep 1: Kewrite the linear programming problem by e%pressing all the constraints in form and transforming them into e'#ations
thro#gh slac) variables.
(tep /: $%hibit the above problem in the form of a simple% tablea#. *f the optimality condition is satisfied and one or more basic
variables have negative val#es, the d#al simple% method is applicable.
(tep .: Ieasibility @ondition: "he basic variable with the most negative val#e becomes the departing variable C.+.!. @all the row
in which this val#e appears the wor) row. *f more than one candidate for C.+. e%ists, choose one.
(tep ,: ?ptimality @ondition: Iorm ratios by dividing all b#t the last element of the last row of cj !j val#es minimi7ation
problem! or the !j cj val#es ma%imi7ation problem! by the corresponding negative coefficients of the wor) row. "he
nonbasic variable with the smallest absol#te ration becomes the entering variable $.+.!. Cesignate this element in the
wor) row as the pivot element and the corresponding col#mn the wor) col#mn. *f more than one candidate for $.+.
e%ists, choose one. *f no element in the wor) row is negative, the problem has no feasible sol#tion.
2&4
(tep : :se elementary row operations to convert the pivot element to 1 and then to red#ce all the other elements in the wor)
col#mn to 7ero.
(tep 2: Kepeat steps . thro#gh  #ntil there are no negative val#es for the basic variables.
1&4
Solved Problems
Problem &
:se the d#al simple% method to solve the following problem.
;inimi7e: ! = 2x1 + x2 + 3x3
s.t. x1 2x2 + x3 4
2x1 + x2 + x3 8
x1 x3 0
<ith: all variables nonnegative
$%pressing all the constraints in the form and adding the slac) variables, the problem becomes:
;inimi7e: ! = 2x1 + x2 + 3x3 + 0x4 + 0x5 + 0x6
s.t. x1 + 2x2 x3 + x4 = 4
2x1 + x2 + x3 + x5 = 8
x1 + x3 + x5 = 0
<ith: all variables nonnegative
4 4 4 4 . 1 / !
4 1 4 4 1 4 1
A 4 1 4 1 1 /
, 4 4 1 1 / 1
2

L
,
2  , . / 1
j j
! c
x
x
x
x x x x x x
(ince all the cj M !j! val#es are nonnegative, the above sol#tion is optimal. 6owever, it is infeasible beca#se it has a nonpositive
val#e for the basic variable x4. (ince x4 is the only nonpositive variable, it becomes the departing variable C.+.!.
. / : ratios absol#te
4 4 1 1 / 1 : row
4 4 4 . 1 / : !
,
2  , . / 1
x
r+ ! c
x x x x x x
j j
(ince x1 has the smallest absol#te ratio, it becomes the entering variable $.+.!. "h#s the element M 1, mar)ed by the asteris),
becomes the pivot element. :sing elementary row operations, we obtain the ne%t tablea#.
A 4 4 / 1  4 : !
, 1 4 1 / / 4
4 4 1 / 1  4
, 4 4 1 1 / 1
2

1
2  , . / 1
j j
! c
x
x
x
x x x x x x
(ince all the variables have nonnegtive val#es, the above optimal sol#tion is feasible. "he optimal and feasible sol#tion is x1 = 4,
x2 = 0, x3 = 0, with ! = 8.
Problem '
:se the d#al simple% method to solve the following problem.
;a%imi7e: ! = 2x1 3x2
s.t. x1 + x2 2
2x1 + x2 10
x1 + x2 8
with: x1 and x2 nonnegative
$%pressing all the constraints in the form and adding the slac) variables, the problem becomes:
;a%imi7e: ! = 2x1 3x2
s.t. x1 + x2 2
2x1 + x2 10
x1 + x2 8
with: all variables nonnegative
A&4
4 4 4 4 . / : !
A 1 4 4 1 1
14 4 1 4 1 /
/ 4 4 1 1 1

,
L
.
 , . / 1
j j
c !
x
x
x
x x x x x
(ince all the cj M !j! val#es are nonnegative, the above sol#tion is optimal. 6owever, it is infeasible beca#se it has a nonpositive
val#e for the basic variable x3. (ince x3 is the only nonpositive variable, it becomes the departing variable C.+.!.
. / : ratios absol#te
4 4 1 1 1 : row
4 4 4 . / : row !
.
 , . / 1
x
c !
x x x x x
j j
(ince x1 has the smallest absol#te ration, it becomes the entering variable $.+.!. "h#s the element M 1, mar)ed by the asteris), is
the pivot element. :sing elementary row operations, we obtain the ne%t tablea#.
, 4 4 / 1 4 : !
2 1 4 1 4 4
2 4 1 / 1 4
/ 4 4 1 1 1

,
1
 , . / 1
j j
c !
x
x
x
x x x x x
(ince all the variables have nonnegative val#es, the above optimal sol#tion is feasible. "he optimal and feasible sol#tion is x1 = /,
x2 = 4, with 7 = ,.
Problems:
...1
;inimi7e: ! = 3x1 9 x2
s.t. 3x1 9 x2 4
x1 M x2 0
with: x1, x2 nonnegative
.../
;inimi7e: ! = 5x1 + 3x2 + x3
s.t. M x1 9 x2 9 x3 2
2x1 9 x2 M x3 4
with: all variables nonnegative
....
;inimi7e: ! = x1 + 2x2 + 3x3
s.t. x1 2x2 + x3 1
3x1 4x2 + 6x3 8
2x1 4x2 + 2x3 2
with: all variables nonnegative
...,
;a%imi7e: ! = 4x1 3x2
s.t. x1 9 x2 2
x1 + 2x2 8
with: x1, x2 nonnegative
...
;a%imi7e: ! = 3x1 4x2 5x3
s.t. x1 + x2 10
x1 + 3x2 + x3 "
x2 + x3 4
with: all variables nonnegative
...2
;inimi7e: ! = 3x1 + "x2
s.t. x1 + 3x2 6
2x1 + 3x2 "
with: x1, x2 nonnegative
3&4
)inear Programming: (uality and Sensitivity #nalysis
$very linear program in the variables x1, x2, J , xn has associated with it another linear program in the variables +1, +2, J , +.
where m is the n#mber of constraints in the original program!, )nown as its d#al. "he original program, called the primal,
completely determines the form of its d#al.
Symmetric (uals
"he d#al of a primal! linear program in the nonstandard! matri% form
minimi7e: ! = &
%
$
1! s.t. '$ (
with: $ 0
is the linear program
ma%imi7e: ! = (
%
/
/! s.t. '
%
/ &
with: / 0
@onversely, the d#al of the second program is the first program.
Nrograms 1! and /! are symmetrical in that both involve nonnegative variables and ine'#ality constraints= the +1, +2, J , +. are
)nown as the s0..etric 12als of each other. "he d#al variables are sometimes called s,a1+ csts.
(ual solutions
!"eorem & *(uality !"eorem+: *f an optimal sol#tion e%ists to either the primal or symmetric d#al program, then the other
program also has an optimal sol#tion and the two obEective f#nctions have the same optimal val#e.
*n s#ch sit#ation, the optimal sol#tion to the primal d#al! is fo#nd in the last row of the final simple% tablea# for the d#al
primal!, in those col#mns associated with the slac) or s#rpl#s variables. (ince the sol#tions to both programs are obtained by
solving either one, it may be comp#tationally advantageo#s to solve a programBs d#al rather than the program itself.
!"eorem ' *Complementary Slackness Principle+: >iven that the pair of symmetric d#als have optimal sol#tions, then if the k&
th constraint of one system holds as an ine'#ality M i.e., the associated slac) or s#rpl#s variable is positive M the k&th
component of the optimal sol#tion of tits symmetric d#al is 7ero.
,nsymmetric (auls
Ior primal programs in standard matri% form, d#als may be defined as follows:
Primal (ual
;inimi7e: ! = &
%
$
s.t. '$ = (
with: O 0
.!
;a%imi7e: ! = (
%
/
s.t. '
%
/ &
,!
;a%imi7e: ! = &
%
$
s.t '$ = (
with: $ 0
!
;inimi7e: ! = (
%
/
s.t. '
%
/ & 2!
@onversely, the d#als of programs ,! and 2! are defined as programs .! and !, respectively. (ince the d#al of a program in
standard form is not itself in standard form, these d#als are 2ns0..etric. "heir forms are consistent with and a direct conse'#ence
of the definition of symmetric d#als.
"heorem 1 is valid for #nsymmetric d#als too. 6owever, the sol#tion to an #nsymmetric d#al is not, in general, immediately
apparent from the sol#tion to the primal= the relationships are
1
0
%
0
%
' & /
L
or
0
1 %
0
& ' /
! L
1!
1 %
0
%
0
%
' ( $
! L
or
0
1
0
( ' $
L
A!
*n 1!, &0 and '0 are made #p of those elements of & and ', in either program .! or !, that correspond to the basic variables in
$L= in A!, (0 and '0 are made #p of those elements of ( and ', in either program ,! or 2!, that correspond to the basic variables
in /L.
Solved Problems
1. Cetermine the symmetric d#al of the program
;inimi7e: ! = 5x1 + 2x2 + x3
s.t. 2x1 + 3x2 + x3 20
6x1 + 8x2 + 5x3 30
7x1 + x2 + 3x3 40
x1 + 2x2 + 4x3 50
with: all variables nonnegative
14&4
ma%imi7e: ! = 20+1 + 30+2 + 40+3 + 50+4
2+1 + 6+2 + 7+3 + +4 5
3+1 + 8+2 + +3 + 2+4 2
+1 + 5+2 + 3+3 + 4+4 1
with: all variables nonnegative
/. Cetermine the symmetric d#al of the program
;a%imi7e: ! = 2x1 + x2
s.t. x1 + 5x2 10
x1 + 3x2 6
2x1 + 2x2 8
with: all variables nonnegative
minimi7e: ! = 10+1 + 6+2 + 8+3
s.t. +1 + +2 + 2+3 2
5+1 + 3+2 + 2+3 1
with: all variables nonnegative
sol#tion for primal and d#al problems:
variables slack
4 4 4 4 1 / : !
A 1 4 4 / /
2 4 1 4 . 1
14 4 4 1  1
L

,
.
 , . / 1
j j
c !
x
x
x
x x x x x
A 1 4 4 1 4
, / 1 4 4 1 1
/ / 1 1 4 / 4
2 / 1 4 1 , 4
1
,
.
 , . / 1
x
x
x
x x x x x
12al t,e t sl2tin
variables s2r3l2s
. 4 4 1 1 , , 2
4 4 4 4 4 A 2 14 !
1 1 4 1 4 / . 
/ 4 1 4 1 / 1 1
1
2
1 2  , . / 1
j j
! c
+
+
+ + + + + + +
A 4 , 4 / 2
1 4 / 1 1 / 1 / 1
1 1 1 4  ,
.

 , . / 1
+
+
+ + + + +
3ri.al t,e t sl2tin
/. Cetermine the d#al of the program
;a%imi7e: ! = x1 + 3x2 2x3
s.t. 4x1 + 8x2 + 6x3 = 25
7x1 + 5x2 + "x3 = 30
with: all variables nonnegative
"he primal program can be solved by the two&phase method if artificial variables %, and %, respectively, are first added to the
left&hand sides of the constraints e'#ations. (imple% tablea#% res#lt.
55 0 0 15 13 11
0 0 0 2 3 1 c !
30 1 0 " 5 7 x
25 0 1 6 8 4 x
x x x x x
j j
5
4
5 4 3 2 1
!
777 7 668 3 0 0
1"3 3 167 1 0 1 x
528 1 1668 0 1 0 x
x x x
1
2
3 2 1
. .
. .
. .
"he primal program has the form != its #nsymmetrid d#al is given by 2! as
;inimi7e: ! = 25+1 + 30+2
s.t. 4+1 + 4+2 1
8+1 + 5+2 2
6+1 + "+2 2
11&4
4 0 0 0 1 1 12 12 12 12
0 0 0 0 0 0 30 30 25 25 ! c
2 0 0 1 0 0 " " 6 6 +
3 1 0 0 1 0 5 5 8 8 +
1 0 1 0 0 1 7 7 4 4 +
+ + + + + + + + +
j j
"
11
10
11 10 " 8 7 6 5 4 3
!
778 7 0 528 1 1"5 3 0 0 0 0
667 3 1 1667 0 167 1 0 0 0 0 +
1111 0 0 1111 0 2222 0 1 1 0 0 +
4444 0 0 1"44 0 138" 0 0 0 1 1 +
+ + + + + + +
"
6
3
" 8 7 6 5 4 3
. . .
. . .
. . .
. . .
+1 = +3 +4 = 0*4444
+2 = +5 +6 = 0*1111
"o verify, we note that the basic variables in $
L
are x1 and x2
[ ] [ ] [ ] [ ] 1111 0 4444 0 36 4 36 16
36 4 36 7
36 8 36 5
3 1
5 7
8 4
3 1 /
1
%
. , . , , ,
L
1
]
1
1
]
1
1
1
1
]
1
Sensitivity #nalysis
"he scope of linear programming does not end at finding the optimal sol#tion to the linear model of a real&life problem.
(ensitivity analysis of linear programming contin#es with the optimal sol#tion to provide additional practical insight of the model.
(ince this analysis e%amines how sensitive the optimal sol#tion is to changes in the coefficients of the LN model, it is called
sensitivity analysis. "his process is also )nown as postoptimality analysis beca#se it starts after the optimal sol#tion is fo#nd.
(ince we live in a dynamic world where changes occ#r constantly, this st#dy of the effects on the sol#tion d#e to changes in the
data of a problem is very #sef#l
*n general, we are interested in finding the effects of the following changes on the optimal LN sol#tion:
i! @hanges in profit5#nit or cost5#nit coefficients! of the obEective f#nction.
ii! @hanges in the availability of reso#rces or capacities of prod#ction5service center or limits on demands
re'#irements vector or K6( of constraints!.
iii! @hanges in reso#rce re'#irements5#nits of prod#cts or activities technological coefficients of variables! in
constraints.
iv! Addition of a new prod#ct or activity variable!.
v! Addition of a new constraint.
"he sensitivity analysis will be disc#ssed for linear programs of the form:
;a%imi7e: ! = &
%
$
s.t. '$ (
with $ 0
where $ is the col#mn vector of #n)nowns= &
%
is the row vector of the corresponding costs cost vector!= ' is the coefficient
matri% of the constraints matri% of technological coefficients!= and ( is the col#mn vector of the right&hand sides of the
constraints re'#irements vector!.
Example &:
;a%imi7e: ! = 20x1 9 10x2
s.t. x1 9 2x2 40
3x1 9 2x2 60
with: x1 and x2 nonnegative
"his program is p#t into the following standard form by introd#cing the slac) variables %. and %,:
;a%imi7e: ! = 20x1 9 10x2 9 0x3 9 0x4
x1 9 2x2 9 x3 = 40
3x1 9 2x2 9 x4 = 60
with: all variables nonnegative
"he sol#tion for this problem is s#mmari7ed as follows:
*nitial (imple% "ablea#:
1/&4
0 0 0 10 20 c !
60 1 0 2 3 0 x
40 0 1 2 1 0 x
0 0 10 20
x x x x
j j
4
3
4 3 2 1
!
Iinal (imple% "ablea#:
400 3 20 0 3 10 0 c !
20 3 1 0 3 2 1 20 x
20 3 1 1 3 4 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1
!
(ince the last row of the above tablea# contains no negative elements, the optimal sol#tion is x1 = 20, x2 = 0, with ! = 400
Example && ;odification of the cost &
%
a! @oefficients of the nonbasic variables.
Let the new val#e of the cost coefficient corresponding to the nonbasic variable x2 be 15 instead of 10.
"he corresponding simple% tablea# is
400 3 20 0 3 5 0 c !
20 3 1 0 3 2 1 20 x
20 3 1 1 3 4 0 0 x
0 0 15 20
x x x x
j j
1
3
4 3 2 1
!
(ince !2 M c2! P 0, the new sol#tion is not optimal. "he reg#lar simple% method is #sed to reoptimi7e the problem, starting with x2
as the entering variable.
"he new optimal tablea# is
425 4 25 4 5 0 0 c !
10 2 1 2 1 0 1 20 x
15 4 1 4 3 1 0 15 x
0 0 15 20
x x x x
j j
1
2
4 3 2 1
!
!
(ince !2 M c2! P 0, the new sol#tion is not optimal. "he reg#lar simple% method is resorted to for reoptimi7ation, first by entering
x2. "he new optimal tablea# is
250 2 5 2 5 0 0 c !
10 2 1 2 1 0 1 20 x
15 4 1 4 3 1 0 10 x
0 0 10 10
x x x x
j j
1
2
4 3 2 1
!
,
_
,
_
,
_
,
_
3
1
3
1
1
3
4 3
3
1 3 0
4 0
3 1 0
3 1 1
x
x
1.&4
(ince x3 P 0, the new sol#tion is not feasible. "he d#al simple% method #sed to clear the infeasibility starting with the following
tablea#:
67 866 67 6 0 33 3 0 c !
33 43 33 0 0 67 0 1 20 x
33 3 33 0 1 33 1 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1
. . . !
. . .
. . .
"he new final tablea# is
800 0 20 30 0 c !
40 0 1 2 1 20 x
10 1 3 4 0 0 x
0 0 10 20
x x x x
j j
1
4
4 3 2 1
!
"he optimal and feasible sol#tion is x1 = 40, x2 = 0, with ! = 800.
Example &. ;odification of the matri% of coefficients '
"he problem becomes more complicated, when the technological coefficients of the basic variables are considered. "his is
beca#se here the matri% #nder the starting sol#tion changes. *n this case, it may be easier to solve the new problem than resort to
the sensitivity analysis approach. "herefore o#r analysis is limited to the case of the coefficients of nonbasic variables only.
Let the technological coefficients of x2 be changed from 2, 2!
%
to 2, 1!
%
.
"hen the new technological coefficients of x2 in the optimal simple% tablea# of the original primal problem are given by
,
_
,
_
,
_
3
1
3
5
1
2
3
1
0
3
1
1
6 4
2
1
6ence the new simple% tablea# becomes
400 3 20 0 3 10 0 c !
20 3 1 0 3 1 1 20 x
20 3 1 1 3 5 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1
!
(ince here !2 M c2! P 0, the new sol#tion is not optimal. Again the reg#lar simple% method is resorted to for reoptimi7ation, first
by entering x2. "he new optimal tablea# is
440 6 2 0 0 c !
16 5 2 5 1 0 1 20 x
12 5 1 5 3 1 0 10 x
0 0 10 20
x x x x
j j
1
2
4 3 2 1
!
,
_
,
_
,
_
3
1
3
8
1
3
3
1
0
3
1
1
6 4
k
1
"he corresponding !k M ck! = 852!0! 9 153!20! M 30 = 7053
"h#s the modified simple% tablea# is
1,&4
400 3 20 0 3 70 3 10 0 c !
20 3 1 0 3 1 3 2 1 20 x
20 3 1 1 3 8 3 4 0 0 x
0 0 30 10 20
x x x x x
j j
1
3
4 3 k 2 1
!
Dow entering the variable xk, the reg#lar simple% method is applied to obtain the following optimal tablea#.
575 8 30 8 70 0 15 0 c !
2 35 8 3 8 1 0 2 1 1 20 x
2 15 8 1 8 3 1 2 1 0 30 x
0 0 30 10 20
x x x x x
j j
1
k
4 3 k 2 1
!
Qy #sing the row operations, the coefficient of x1 in the new constraint is made 7ero. "he modified tablea# becomes
400 0 3 20 0 3 10 0 c !
10 1 3 2 0 3 5 0 0 x
20 0 3 1 0 3 2 1 20 x
20 0 3 1 1 3 4 0 0 x
0 0 0 10 20
x x x x x
j j
5
1
3
5 4 3 2 1
! +
"he d#al simple% method is #sed to overcome the infeasibility by departing the variable x5. "he new tablea# is
380 2 8 0 0 0 c !
6 5 3 5 2 0 1 0 10 x
16 5 2 5 3 0 0 1 20 x
12 5 4 5 1 1 0 0 0 x
0 0 0 10 20
x x x x x
j j
2
1
3
5 4 3 2 1
! +
"he above tablea# gives the optimal and feasible sol#tion as x1 = 16, x2 = 6, x3 = 12, with ! = 380.
!"e 1evised Simplex et"od
@onsider the following linear programming problem in standard matri% form:
;a%imi7e: ! = &
%
$
s.t. '$ = (
with: $ 0
where $ is the col#mn vector of #n)nowns, incl#ding all slac), s#rpl#s, and artificial variables= &
%
is the row vector of
corresponding costs= ' is the coefficient matri% of the constraint e'#ations= and ( is the col#mn vector of the right&hand side of the
constraint e'#ations. "hey are represented as follows:
,
_
,
_
,
_
,
_
,
_
.n 21 1 .
n 2 22 21
n 1 12 11
n
2
1
n
2
1
n
2
1
a a a
a a a
a a a
'
0
0
0
0
(
(
(
(
c
c
c
&
x
x
x
$
, , , ,
1&4
Let $s = the col#mn vector of basic variables, &s
%
= the row vector of costs corresponding to $s, and 4 = the basis matri%
corresponding to $s.
(tep 1: $ntering +ector 6k:
Ior every nonbasic vector 6j, calc#late the coefficient
!j M cj = /6j M cj ma%imi7ation program!, or cj M !j = cj M /6j minimi7ation program!, where / = &s
%
4
51
.
"he nonbasic vector 6j with the most negative coefficient becomes the entering vector $.+.!, 6k.
*f more than one candidate for $.+. e%ists, choose one.
(tep /: Ceparting +ector 6r:
a! @alc#late the c#rrent basic $
s
: $s = 4
51
(
b! @orresponding to the entering vector 6k, calc#late the constraint coefficients t): tk = 4
51
6k
c! @alc#late the ratio 7:
. 2 1 i 0 t
t
$
ik
ik
i s
i
, , , , ,
!
min
'
>
"he departing vector C.+.!, 6r, is the one that satisfies the above condition.
Dote: *f all tkj P 0, there is no bo#nded sol#tion for the problem. (top.
(tep .: Dew Qasis:
4
51
ne+ = 84
51
, where 8 = 21, J, 2r51, 9, 2r+1, J, 2.!
Dote,
'
,
_
r i if
t
1
r i if
t
t
+,ere
rk
rk
ik
.
2
1
,
,
,
and 2i is a col#mn vector with 1 in the ith element and 0 in the other . M 1! elements.
(et 4
51
= 4
51
ne+ and repeat steps 1 thro#gh ., #ntil the following optimality condition is satisfied.
!j M cj 0 ma%imi7ation problem!, or cj M !j 0 minimi7ation problem!
"hen the optimal sol#tion is as follows: $s = 4
51
(= ! = &s
%
$s
Solved Problems
1. :se revised simple% method to solve the following problem.
;a%imi7e: ! = 10x1 9 11x2
s.t. x1 + 2x2 150
3x1 + 4x2 200
6x1 + x2 175
with: x1 and x2 nonnegative
"his program is p#t in standard form by introd#cing the slac) variables x3, x4, and x5,
;a%imi7e: ! = 10x1 9 11x2 + 0x3 + 0x4 + 0x5
s.t. x1 + 2x2 9 x3 = 150
3x1 + 4x2 9 x4 = 200
6x1 + x2 9 x5 = 175
<ith all variables nonnegative
61 =
,
_
6
3
1
, 62 =
,
_
1
4
2
, 63 =
,
_
0
0
1
, 64 =
,
_
0
1
0
, 65 =
,
_
1
0
0
, ( =
,
_
175
200
150
:nitiali!atin;
$s = x3, x4, x5!
"
= &s
%
= 0, 0, 0!
4 = 63, 64, 65! = : = 4
51
=
,
_
1 0 0
0 1 0
0 0 1
:teratin <1;
"he nonbasic vectors are 61 and 62.
a! $ntering +ector: / = &s
%
4
51
= 0, 0, 0!: = 0, 0, 0!
!1 M c1, !2 M c2! = /61, 62! M c1, c2! = 0, 0, 0!
,
_
1 6
4 3
2 1
M 10, 11! = M10, M11!
12&4
(ince the most negative coefficient corresponds to 62, it becomes the entering vector $.+.!
b! Ceparting +ector:
$s = 4
51
( = :( = ( =
,
_
175
200
150
t2 = 4
&1
62 = :62 = 62 =
,
_
1
4
2
50
1
175
4
200
2
150
'
, , min
(ince the minim#m ratio corresponds to 64, it becomes the departing vector C.+.!
c! Dew Qasis:
,
_
,
_
,
_
4 1
4 1
2 1
4 1
4 1
4 2
t
t
t
1
t
t
42
52
42
42
32
= $ = 21, 9, 23!
,
_
1 , 1 4
4 , 1 4
4 / 1 1
1 1
8 8: 84 4
ne+
(#mmary of *teration R1:
$s = x3, x2, x5!
%
= &s
%
= 0, 11, 0!
:teratin <2;
Dow the nonbasic vectors are 61 and 64
a! $ntering +ector:
/ = &s
%
4
51
=
( ) ( ) 0 4 11 0
1 4 1 0
0 4 1 0
0 2 1 1
0 11 0 , , , ,
,
_
,
_
0 6
1 3
0 1
M 10, 0! = M15,, 11)4!
(ince the most negative coefficient corresponds to 61, it becomes the entering vector $.+.!
b! Ceparting +ector:
,
_
,
_
,
_
1/
4
4
11
/44
14
1 , 1 4
4 , 1 4
4 / 1 1
1
( 4 $
s
,
_
,
_
,
_
, /1
, .
/ 1
2
.
1
1 , 1 4
4 , 1 4
4 / 1 1
1
1
1
6 4 t
/1 44
, /1
1/
,
, .
4
, min
'
(ince the minim#m ratio corresponds to 65, it becomes the departing vector C.+.!.
11&4
c! Dew Qasis:
,
_
,
_
,
_
/1 ,
1 1
/1 /
, /1
1
, /1
, .
, /1
/ 1
1
1
1
/1
/
.1
t
t
t
t
t
= $ = 21, 22, 9!
,
_
,
_
,
_
21 4 21 1 0
7 1 7 2 0
21 2 21 11 1
1 4 1 0
0 4 1 0
0 2 1 1
21 4 0 0
7 1 1 0
21 2 0 1
84 4
1 1
ne+
(#mmary of *teration R/:
$s = x3, x2, x1!
%
= &s
%
= 0, 11, 10!
:teratin <3;
Dow the nonbasic vectors are 65 and 64.
a! $ntering +ector:
( ) ( ) . 1 , . A , 4
/1 , /1 1 4
1 1 1 / 4
/1 / /1 11 1
14 , 11 , 4
1
,
_
4 & /
%
s
!5 M c5, !4 M c4! = /65, 64! M c5, c4! = 0, 8)3 , 1)3!
,
_
4 1
1 4
4 4
M 0, 0! = 153, 8)3!
(ince all the coefficients are nonnegative, the above step gives the optimal basis. "he optimal val#es of the variables and the
obEective f#nction are as follows:
,
_
,
_
,
_
,
_
/1 44
1 //
/1 1.44
11
/44
14
/1 , /1 1 4
1 1 1 / 4
/1 / /1 11 1
1
1
/
.
( 4
x
x
x
( ) . 111
/1 44
1 //
/1 1.44
14 , 11 , 4
,
_
s
%
s
$ & !
/. :se the revised simple% method to solve the following problem.
;inimi7e: ! = 3x1 + 2x2 + 4x3 + 6x4
s.t. x1 + 2x2 + x3 + x4 1000
2x1 + x2 + 3x3 + 7x4 1500
with: all variables nonnegative
"his program is p#t in standard form by introd#cing the s#rpl#s variables x5 and x7, and the artificial variables x6 and x8.
;inimi7e: ! = 3x1 + 2x2 + 4x3 +6x4 + 0x5 + x6 + 0x7 + x8
s.t. x1 + 2x2 + x3 + x4 x5 + x6 = 1000
2x1 + x2 + 3x3 + 7x4 x7 + x8 = 1500
with: all variables nonnegative
,
_
,
_
,
_
,
_
,
_
,
_
,
_
,
_
,
_
1500
1000
(
1
0
6
1
0
6
0
1
6
0
1
6
7
1
6
3
1
6
1
2
6
2
1
6
8 7 6 5 4 3 2 1
, , , , , , , ,
:nitiali!atin;
$s = =x6, x8>
%
? &s
%
= =, >
( )
,
_
1 0
0 1
4 : 6 6 4
1
8 6
,
:teratin <1;
"he nonbasic vectors are 61, 62, 63, 64, 65, and 67.
a! $ntering +ector:
1A&4
( ) ( ) : 4 & /
1 %
s
, ,
( ) ( ) ( )
7 5 4 3 2 1 7 5 4 3 2 1 7 7 5 5 4 4 3 3 2 2 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )
,
_
1 0 7 3 1 2
0 1 1 1 2 1
0 0 6 4 2 3 , , , , , ,
( ) 6 8 4 4 2 3 3 3 , , , , , + + + +
(ince the most negative coefficient corresponds to 64, it becomes the entering vector $.+.!.
b! Ceparting +ector:
,
_
1500
1000
( :( ( 4 $
1
s
,
_
7
1
6 :6 6 4 t
4 4 4
1
4
{ } 7 1500 7 1500 1000 , min
(ince the minim#m ratio corresponds to 68, it becomes the departing vector C.+!
c! Dew Qasis:
=
,
_
,
_
7 1
7 1
t
1
t
t
84
84
64
8 = =21, 9>
,
_
7 1 0
7 1 1
8 8: 84 4
1 1
ne+
(#mmary of *teration R1:
( ) ( ) 6 & x x $
%
s
%
4 6 s
, = ,
:teratin <2;
Dow the nonbasic vectors are 61, 62, 63, 68, 65, and 67.
a! $ntering +ector:
( ) ( ) 7 7 6
7 1 0
7 1 1
6 4 & /
1 %
s
,
_
, ,
( ) ( ) ( )
7 5 8 3 2 1 7 5 8 3 2 1 7 7 5 5 8 8 3 3 2 2 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )
,
_
1 0 1 3 1 2
0 1 0 1 2 1
7 7 6 0 0 4 2 3 , , , , , ,
( ) 7 6 7 7 6 7 8 7 10 7 4 7 8 7 13 7 " 7 5 + + + + , , , , ,
(ince the most negative coefficient corresponds to 62, it becomes the entering vector $.+.!.
b! Ceparting +ector:
( ) ( ) 7 1500 7 5500 7 1500 7 1500 1000
1500
1000
7 1 0
7 1 1
( 4 $
1
s
, ,
,
_
,
_
,
_
,
_
,
_
7 1
7 13
1
2
7 1 0
7 1 1
6 4 t
2
1
2
3
5500
1500
3
5500
7 1
7 1500
7 13
7 5500
'
'
, min , min
(ince the minim#m ratio corresponds to 66, it becomes the departing vector C.+.!
c! Dew Qasis:
( )
2
62
42
62
2 8
13 1
13 7
7 13
7 1
7 13
1
t
t
t
1
, =
,
_
,
_
,
_
,
_
,
_
,
_
13 2 13 1
13 1 13 7
7 1 0
7 1 1
1 13 1
0 13 7
84 4
1 1
ne+
13&4
(#mmary of *teration R/:
( ) ( ) 6 2 & x x $
%
s
%
4 2 s
, = ,
:teratin <3;
Dow the nonbasic vectors are 61, 66, 63, 68, 65, and 67.
a! $ntering +ector:
( ) ( ) 13 10 13 8
13 2 13 1
13 1 13 7
6 2 4 & /
1 %
s
, ,
,
_
( ) ( ) ( )
7 5 8 3 6 1 7 5 8 3 6 1 7 7 5 5 8 8 3 3 6 6 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )
,
_
1 0 1 3 0 2
0 1 0 1 1 1
13 10 13 8 0 0 4 3 , , , , , ,
( ) 13 10 13 8 13 10 13 14 13 8 13 11 , , , , , + +
(ince all the coefficients are nonnegative, the above step gives the optimal basis. "he optimal val#es of the variables and the
obEective f#nction are as follows:
,
_
,
_
,
_
,
_
13 2000
13 5500
1500
1000
13 2 13 1
13 1 13 7
( 4
x
x
1
4
2
( ) 13 23000
13 2000
13 5500
6 2 $ & !
s
%
s
,
_
,
2armarkar3s #lgorit"m
"he simple% method of linear programming finds the optim#m sol#tion by starting at the origin and moving along the adEacent
corner points of the feasible sol#tions space. Altho#gh this techni'#e has been s#ccessf#l is solving LN problems of vario#s si7es,
the n#mber of iterations becomes prohibitive for some h#ge problems. "his is an e%ponential time algorithm.
?n the other hand, Sarmar)arBs interior point Lp algorithm is a polynomial time algorithm. "his new approach finds the optim#m
sol#tion by starting at a trial sol#tion and shooting thro#gh the interior of the feasible sol#tion space. Altho#gh this proEective
algorithm may be advantageo#s in solving very large LN problems, it becomes very c#mbersome for not&so&large problems.
@onsider the following form of the linear programming problem:
;inimi7e: ! = &
%
$
s.t. '$ = 0
1$ = 1
$ 0
where $ is a col#mn vector of si7e n= & is an integer col#mn vector of si7e n= 1 is a #nit row vector of si7e n= ' is an integer matri%
of si7e . T n!= n /.
*n addition, ass#me the following two conditions:
1. $0 = 15n, ..., 15n! is a feasible sol#tion.
/. minim#m ! = 0
Summary o$ 2armarkar3s Iterative Procedure
1. Nreliminary (tep:
k = 4
$0 = 15n, ..., 15n!
%
! 1
1
n n
r
n n . ! 1
/. *teration k:
a! Cefine the following:
i! @0 = $0
ii! Ak = diagU$kV, which is the diagonal matri% whose diagonal consists of the elements of $k.
iii!
,
_
1
k
'A
6
iv!
k
%
A & &
b! @omp#te the following:
i!
% % %
3
& 6 66 6 : & H ! G
1
Dote: *f &3 = 0, any feasible sol#tion becomes an optimal sol#tion. (top.
ii!
3
3
ne+
&
&
r @ @
4
iii! ! 1 !
1 ne+ k ne+ k k
@ A @ A $
+
iv! ! = &
%
$k+1
/4&4
v! k = k 9 1
vi! Kepeat iteration k #ntil the obEective f#nction !! val#e is less than a prescribed tolerance B.
!rans$orming a )inear Programming Problem into 2armarkar3s Special Form
@onsider the following linear program in the matri% form:
;inimi7e: ! = &
%
$
s.t. '$ (
with $ 0
"he simplified steps of converting the above problem into Sarmar)arBs special form are as follows:
1. (ince every primal linear program in the variables x1, x2, C,xn has associated with it a d#al linear program in the variables +1,
+2, C, +. where . is the n#mber of constraints in the original program!, write the d#al of the given primal problem as shown
below:
;a%imi7e: ! = (
%
/
s.t. '
%
/ &
with / 0
/. a! *ntrod#ce slac) and s#rpl#s variables to the primal and d#al problems.
b! @ombine primal and d#al problems.
.. a! *ntrod#ce a bo#nding constraint
+ D + x
i i
D sho#ld be s#fficiently large to incl#de all feasible sol#tions of the original problem.!
b! *ntrod#ce a slac) s in the bo#nding constraint and obtain the following e'#ation:
+ + D s + x
i i
,. a! *ntrod#ce a d#mmy variable 1 s#bEect to the condition 1 = 1! to homogeni7e the constraints.
b! Keplace the e'#ations
+ + D s + x
i i
and 1 = 1
with the following e'#ivalent e'#ations:
+ + 4 D1 s + x
i i
and
+ + + + ! 1 D 1 s + x
i i
. *ntrod#ce the following transformations so as to obtain one on the K6( of the last e'#ation:
xj = D 9 1!0j, j = 1, 2, ..., . 9 n
+j = D 9 1!0.9n9j, j = 1, 2, ..., . 9 n
s = D 9 1!02.+2n+1
1 = D 9 1!02.+2n+2
2. *ntrod#ce an artificial variable 02.+2n+2 to be minimi7ed! in all the e'#ations s#ch that the s#m of the coefficients in each
homogeneo#s e'#ation is 7ero and the coefficient of the artificial variable in the last e'#ation is one.
(olved Nroblems
1. @arry o#t the first two iterations of Sarmar)arBs algorithm for the following problem.
;inimi7e: ! = 2x2 x3
s.t. x1 2x2 + x3 = 0
x1 + x2 + x3 = 1
with all variables nonnegative
Nreliminary (tep:
k = 0
$0 = =1)n, C, 1)n>
%
= =1)3, 1)3, 1)3>
%
( ) ( ) 6 1 1 3 3 1 1 n n 1 r
( ) ( ) ( ) " 2 3 3 1 3 n 3 1 n
*teration 4:
@0 = $0 = =1)3, 1)3, 1)3>
%
A0 = 1iaEF$0G = 1iaEF1)3, 1)3, 1)3G
' = =1, 2, 1>? &
%
= =0, 2, 1>
'A0 = =1, 2, 1> 1iaEF1)3, 1)3, 1)3G = F1)3, 2)3, 1)3G
,
_
,
_
1 1 1
3 1 3 2 3 1
1
'A
6
0
( ) { } ( ) 3 1 3 2 0 3 1 3 1 3 1 1iaE 1 2 0 A & &
0
%
, , , , , ,
/1&4
( )
,
_
,
_
333 0 0
0 5 1
66
3 0
0 667 0
66
1
% %
.
.
=
.
( )
,
_
5 0 0 5 0
0 1 0
5 0 0 5 0
6 66 6
1
% %
. .
. .
( ) ( )
% %
1
% %
6
167 0 0 167 0 & 6 66 6 : & . , , .
1
]
1
( ) ( ) 2362 0 167 0 0 167 0 &
2 2
6
. . . + +
( )
( )
( ) ( )
% % %
6
6
0 ne+
3"74 0 3333 0 26"2 0 167 0 0 167 0
2362 0
6 1 " 2
3 1 3 1 3 1
&
&
r @ @ . , . , . . , , .
.
, ,
( )
%
ne+ 1
3"74 0 3333 0 26"2 0 @ $ . , . , .
( )( ) 26"2 0 3"74 0 3333 0 26"2 0 1 2 0 $ & !
%
1
%
. . , . , . , ,
k = 0 + 1 = 1
*teration 1:
{ } { } 3"74 0 3333 0 26"2 0 1iaE $ 1iaE A
1 1
. , . , .
( ) { } ( ) 3"74 0 6666 0 0 3"74 0 3333 0 26"2 0 1iaE 1 2 0 &A &
1
. , . , . , . , . , ,
( ) { } ( ) 3"74 0 6666 0 26"2 0 3"74 0 3333 0 26"2 0 1iaE 1 2 1 'A
1
. , . , . . , . , . , ,
,
_
,
_
1 1 1
3"74 0 6666 0 26"2 0
1
'A
6
1
. . .
( )
,
_
,
_
333 0 0
0 482 1
66
3 0
0 675 0
66
1
% %
.
.
=
.
( )
,
_
( ) ( ) ( ) 2014 0 132 0 018 0 151 0 &
2 2 2
6
. . . . + +
( )
( )
( ) ( )
% % %
6
6
0 ne+
3"28 0 3414 0 2653 0 132 0 018 0 151 0
2014 0
6 1 " 2
3 1 3 1 3 1
&
&
r @ @ . , . , . . , . , .
.
, ,
{ }( ) ( )
% %
ne+ 1
1561 0 1138 0 0714 0 3"28 0 3414 0 2653 0 3"74 0 3333 0 26"2 0 1iaE @ A . , . , . . , . , . . , . , .
3413 0 @ A 1
ne+ 1
.
( )
%
ne+ 1
ne+ 1
2
4574 0 3334 0 20"2 0
@ A 1
@ A
$ . , . , .
( )( ) 20"4 0 4574 0 3334 0 20"2 0 1 2 0 $ & !
%
2
%
. . , . , . , ,
/. @arry o#t the first two iterations of Sarmar)arBs algorithm for the following problem.
;inimi7e: ! = 2x1 + x2 + 2x3 2x4
s.t. 2x1 + x2 + 2x3 2x4 3x5 = 0
2x1 x3 + x4 2x5 = 0
x1 + x2 + x3 + x4 + x5 = 0
with: all variables nonnegative
Nreliminary (tep:
k = 0
$0 = =1)n, C, 1)n>% = =1)5, 1)5, 1)5, 1)5, 1)5>
%
( ) ( ) 20 1 1 5 5 1 1 n n 1 r
( ) ( ) ( ) 15 4 3 5 1 5 n 3 1 n
*teration 4:
@0 = $0 = =1)n, C, 1)n>% = =1)5, 1)5, 1)5, 1)5, 1)5>
%
//&4
A0 = 1iaEU$0V = 1iaEU1)5, 1)5, 1)5, 1)5, 1)5V
( ) 0 2 2 1 2 &
2 1 1 0 2
3 2 2 1 2
'
%
, , , , =
,
_
{ }
,
_
,
_
4 0 4 0 2 0 0 4 0
6 0 4 0 4 0 2 0 4 0
5 1 5 1 5 1 5 1 5 1 1iaE
2 1 1 0 2
3 2 2 1 2
'A
0
. . . .
. . . . .
, , , ,
,
_
,
_
1 1 1 1 1
4 0 2 0 2 0 0 4 0
6 0 4 0 4 0 2 0 4 0
1
'A
6
0
. . . .
. . . . .
( ) { } ( ) 0 4 0 4 0 2 0 4 0 5 1 5 1 5 1 5 1 5 1 1iaE 0 2 2 1 2 A & &
0
%
, . , . , . , . , , , , , , , ,
( )
,
_
,
_
2 0 0 0
0 "8"1 2 8152 0
0 8152 0 3587 1
66
5 0 0
0 4 0 24 0
0 24 0 88 0
66
1
% %
.
. .
. .
= . .
. .
( )
,
_
( ) ( ) ( ) ( ) ( ) 283" 0 1343 0 1526 0 0874 0 0613 0 167 0 &
2 2 2 2 2
3
. . . . . . + + + +
( )
( )( )
( )
% %
3
3
0 ne+
1343 0 1526 0 0874 0 0613 0 167 0
283" 0
20 1 15 4
5 1 5 1 5 1 5 1 5 1
&
&
r @ @ . , . , . , . , .
.
, , , ,
( )
%
1718 0 2321 0 2184 0 212" 0 164" 0 . , . , . , . , .
( )
%
ne+ 1
1718 0 2321 0 2184 0 212" 0 164" 0 @ $ . , . , . , . , .
( )( ) 5154 0 1718 0 2321 0 2184 0 212" 0 164" 0 0 2 2 1 2 $ & !
%
1
%
. . , . , . , . , . , , , ,
k = 0 + 1 = 1
*teration 1:
{ } { } 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE $ 1iaE A
1 1
. , . , . , . , .
( ) { } ( ) 0 4642 0 4368 0 212" 0 32"8 0 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE 0 2 2 1 2 A & &
1
%
, . , . , . , . . , . , . , . , . , , , ,
{ }
,
_
1718 0 2321 0 2184 0 212" 0 164" 0 1iaE
2 1 1 0 2
3 2 2 1 2
'A
1
. , . , . , . , .
,
_
,
_
,
_
1 1 1 1 1
3436 0 2321 0 2184 0 0 32"8 0
5154 0 4642 0 4368 0 212" 0 32"8 0
1
'A
6
1
. . . .
. . . . .
( )
,
_
,
_
2 0 0 0
0 123" 3 3128 0
0 3128 0 242 1
66
5 0 0
0 3284 0 0827 0
0 0827 0 826 0
66
1
% %
.
. .
. .
= . .
. .
( )
,
_
( ) ( ) ( ) ( ) ( ) 2374 0 10"3 0 1214 0 0858 0 0446 0 1425 0 &
2 2 2 2 2
3
. . . . . . + + + +
/.&4
( )
( )( )
( )
% %
3
3
0 ne+
10"3 0 1214 0 0858 0 0446 0 1425 0
2374 0
20 1 15 4
5 1 5 1 5 1 5 1 5 1
&
&
r @ @ . , . , . , . , .
.
, , , ,
( )
%
1725 0 2305 0 2216 0 2112 0 1642 0 . , . , . , . , .
{ }( )
%
ne+ 1
1725 0 2305 0 2216 0 2112 0 1642 0 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE @ A . , . , . , . , . . , . , . , . , .
( )
%
02"6 0 0535 0 0484 0 0450 0 0271 0 . , . , . , . , .
2036 0 @ A 1
ne+ 1
.
( )
%
ne+ 1
ne+ 1
2
1456 0 2628 0 2377 0 220" 0 1330 0
@ A 1
@ A
$ . , . , . , . , .
( )( ) 4367 0 1456 0 2628 0 2377 0 220" 0 1330 0 0 2 2 1 2 $ & !
%
2
%
. . , . , . , . , . , , , ,
.. @onvert the following problem into Sarmar)arBs special from:
;a%imi7e: ! = 2x1 + x2
s.t. x1 + x2 5
x1 x2 3
with: all variables nonnegative
i! C#al of the given problem.
;inimi7e: ! = 5+1 + 3+2
s.t. +1 + +2 2
+1 +2 1
with: all variables nonnegative
ii! *ntrod#ction of slac) and s#rpl#s variables and combination of primal and d#al problems.
x1 + x2 + x3 = 5 +1 + +2 +3 = 2
x1 x2 + x4 = 3 +1 +2 +4 = 1
2x1 + x2 = 5+1 + 3+2
with: all variables nonnegative
iii! Addition of a bo#ndary constraint with slac) variable s.
x1 + x2 + x3 = 5 +1 + +2 +3 = 2
x1 x2 + x4 = 3 +1 +2 +4 = 1
2x1 + x2 5+1 3+2 = 0
+ +
4
1 i
4
1 i
i i
D s + x
with: all variables nonnegative
iv! 6omogeni7ed e'#ivalent system with d#mmy variable 1.
x1 + x2 + x3 51 = 0 +1 + +2 +3 21 = 0
x1 x2 + x4 31 = 0 +1 +2 +4 1 = 0
2x1 + x2 5+1 3+2 = 0
+ +
4
1 i
4
1 i
i i
0 D1 s + x
( )
+ + + +
4
1 i
4
1 i
i i
1 D 1 s + x
with: all variables nonnegative
v! *ntrod#ction of the transformations.
xj = =D + 1>0j, j = 1, C, 4 s = =D + 1>"
+j = =D + 1>04+j, j = 1, C, 4 1 = =D + 1>10
"he above transformations yield the following system:
01 + 02 + 03 5010 = 0 05 + 06 07 2010 = 0
01 02 + 04 3010 = 0 05 06 08 010 = 0
201 + 02 505 306 = 0
0 D0 0
10
"
1 i
i
1 0
10
1 i
i
+
"
1 i
11 10 i
0 0 " D D0 0
11
1 i
i
1 0
with: all variables nonnegative
,. @onvert the following problem into Sarmar)arBs special from:
;a%imi7e: ! = x1 + 4x2
s.t. x1 + 2x2 10
2x1 + 3x2 20
x1 6
with: all variables nonnegative
i! C#al of the give problem.
;inimi7e: ! = 10+1 + 20+2 + 6+3
s.t. +1 + 2+2 + +3 1
2+1 + 3+2 4
with: all variables nonnegative
ii! *ntrod#ction of slac) and s#rpl#s variables and combination of primal and d#al problems.
x1 + 2x2 + x3 = 10 +1 + 2+2 + +3 +4 = 1
2x1 + 3x2 + x4 = 20 2+1 + 3+2 +5 = 4
x1 + x5 = 6
x1 + 4x2 = 10+1 + 20+2 + 6+3
with: all variables nonnegative
iii! Addition of a bo#ndary constraint with slac) variable s.
x1 + 2x2 + x3 = 10 +1 + 2+2 + +3 +4 = 1
2x1 + 3x2 + x4 = 20 2+1 + 3+2 +5 = 4
x1 + x5 = 6
x1 + 4x2 10+1 20+2 6+3 = 0
+ +
5
i
i
5
i
i
D s + x
with: all variables nonnegative
iv! 6omogeni7ed e'#ivalent system with d#mmy variable 1.
x1 + 2x2 + x3 101 = 0 +1 + 2+2 + +3 +4 1 = 0
2x1 + 3x2 + x4 201 = 0 2+1 + 3+2 +5 41 = 0
x1 + x5 61 = 0
x1 + 4x2 10+1 20+2 6+3 = 0
+ +
5
i
i
5
i
i
0 D1 s + x
( )
+ + + +
5
i
i
5
i
i
1 D 1 s + x
with: all variables nonnegative
v! *ntrod#ction of the transformations
xj = =D + 1>0j, j = 1, C, 5 s = =D + 1>11
+j = =D + 1>05+j, j = 1, C, 5 1 = =D + 1>12
"he following transformations yield the following system:
01 + 202 + 03 10012 = 0 06 + 207 + 08 0" 012 = 0
201 + 302 + 04 20012 = 0 206 + 307 010 4012 = 0
01 + 05 6012 = 0
01 + 402 1006 2007 608 = 0
0 D0 0
12
11
i
i
1 0
12
i
i
1 0
13
i
i
x
x
x x x x x
"he noninteger assignment for %. came from the first row of the tablea#, which represents the constraint
/
11
/
1
.
1
/
1
/ , . 1
+ + x x x x
(tep/: Kewrite each fractional coefficient and constant in the constraint e'#ation obtained form (tep1 as the s#m of an integer and
a positive fraction between 4 and 1. "hen rewrite the e'#ation so that the left&hand side contains only terms with fractional
coefficients and a fractional constant!, while the right&hand side contains only terms with integral coefficients and an
integral constant!. $'#ation becomes
/
1

/
1
4
.
/
.
/
1
1
 , . 1
+
,
_
+ +
,
_
+ + +
,
_
+ x x x x
or
, . 1  , 1
. 
/
1
/
1
.
/
/
1
x x x x x x + + + +
(tep.: Ke'#ire the left&hand side of the rewritten e'#ation to be nonnegative. "he res#lting ine'#ality is the new constraint.
4
/
1
/
1
.
/
/
1
 , 1
+ + x x x or
/
1
/
1
.
/
/
1
/ , 1
+ + x x x
Solved Problem
ma%imi7e: ! = 2x1 + x2
s.t. 2x1 + 5x2 17 1!
3x1 + 2x2 10
with: x1 and x2 nonnegative and integral
*gnoring the integer re'#irements and applying the simple% method to the res#lting linear program, we obtain the optimal tablea#
after one iteration.
. /4 . / 4 . 1 4
. 14 . 1 4 . / 1
. .1 . / 1 . 11 4
1
.
, . / 1
x
x
x x x x
"ablea# 1
"he first appro%imation to program, therefore, is x1 = 10)3, x3 = 31)3, x2 = x4 = 0. Qoth x1 and x3 are noninteger. Arbitrary
selecting x1, we consider the constraint represented by the second row of the tablea#, the row defining x1= namely,
.
14
.
1
.
/
, / 1
+ + x x x
<riting each fraction as the s#m of an integer and a fraction between 4 and 1, we have
.
1
.
.
1
4
.
/
4
, / 1
+
,
_
+ +
,
_
+ + x x x or
1 , /
.
.
1
.
1
.
/
x x x +
Ke'#iring the left&hand side of this e'#ation to be nonnegative, we obtain
4
.
1
.
1
.
/
, /
+ x x or 1 /
, /
+x x
as the new constraint. Kewriting the constraints of the original program in the forms s#ggested by the tablea# and adding the new
constraint, we generate the new program
ma%imi7e: ! = 2x1 + x2 + 0x3 + 0x4
s.t.
.
11
x2 + x3
.
/
x4 =
.
.1
/!
x1 +
.
/
x2 +
.
1
x4 =
.
14
2x2 + x4 1
with: all variables nonnegative and integral
/A&4
A s#rpl#s variable, x5, and an artificial variable, x6, are introd#ced into the ine'#ality constraint of /! and then the two&phase
method is applied, with x1, x3, and x6 as the initial set of basic variables. "he optimal simple% tablea# is obtained after only one
iteration. "he first appro%imation to program /! is th#s x1 = 3, x2 = 1)2, x3 = 17)2, x4 = x5 = 0. @hoosing x2 to generate the new
constraint, we obtain from the third row of "ablea# /
/ 1. 2 1 / 1 4 4 4
/ 1 / 1 / 1 4 1 4
. . 1 4 4 4 1
/ 11 2 11 /  1 4 4
/
1
.
 , . / 1
x
x
x
x x x x x
"ablea# /
4
/
1
/
1
/
1
 ,
+ x x or 1
 ,
+x x
"his, combined with the constraints of program /! in the forms s#ggested by the "ablea# /, gives the new integer program
ma%imi7e: ! = 2x1 + x2 + 0x3 + 0x4 + 0x5
s.t. x3
/

x4 +
/

x5 =
/
11
x1 +
.
1
x5 = 3 .!
x2 +
/
1
x4
/
1
x5=
/
1
x4 + x5 1
with: all variables nonnegative and integral
*gnoring the integer constraint and applying the two&phase method to program .!, with x1, x2, x3, and x7 artificial! as the initial
basic set, we obtain the optimal "ablea# ..
. 13 2 1 4 . 1 4 4 4
1 1 1 1 4 4 4
1 / 1 4 1 4 1 4
. A . 1 4 . 1 4 4 1
. /4 2 11 4 . 1. 1 4 4

/
1
.
2  , . / 1
x
x
x
x
x x x x x x
"ablea# .
A new iteration of the process is started from x1 =
.
A
in "ablea# .. "his res#lts in a program whose sol#tion is integral, with x1 =
3, x2 = 0, and ! = 6. "his sol#tion is then the optimal sol#tion to integer program 1!.
Summary o$ t"e Gomory cut algorit"m
@onsider the optimal tablea# that res#lts from applying the simple% method to an integer program with the integer re'#irements
ignored, and ass#me that one of the basic variables, xb is nonintegral. "he constraint e'#ation corresponding to the tablea# row
that determined xb m#st have the form
+
4
0 x 0 x
j j b
1!
where the s#m is over all nonbasic variables. "he 0&terms are the coefficients and the constant term appearing in the tablea# row
determining xb. (ince xb is obtained from 1! by setting the nonbasic variables e'#al to 7ero, it follows that 00 is also nonintegral.
<rite each 0&term in 1! as the s#m of an integer and a nonnegative fraction less than 1:
j j j
f i 0 +
and
4 4 4
f i 0 +
(ome of the fj may be 7ero, b#t f0 is g#aranteed to be positive. $'#ation 1! becomes
( )
+ + +
4 4
f i x f i x
j j j b
or
+
j j j j b
x f f i x i x
4 4
/!
*f each %&variable is re'#ired to be integral, then the left&hand side of /! is integral, which forces the right&hand side also to be
integral. Q#t, since each fj and xj is nonnegative, so too is
j j
x f . "he right&hand side of /! then is an integer which is
smaller than a positive fraction less than 1= that is, a nonpositive integer.
4
4 j j
x f f or
4
4
f x f
j j
"his is the new constraint in the >omory algorithm.
/3&4
#not"er cut algorit"m
@onsider 1! in the s#mmary of >omory algorithm. *f each nonbasic variable xj is 7ero, then xb = 00 is nonintegral. *f xb is to
become integer&val#ed, then at least one of the nonbasic xj m#st be made different from 7ero. (ince all variables are re'#ired to be
nonnegative and integral, it follows that at least one nonbasic variable m#st be made greater than or e'#al to 1. "his in t#rn
implies that the s#m of all the nonbasic variables m#st be made greater that or e'#al to 1. *f this condition is #sed as the new
constraint to be adEoined to the original integer program, we have the c#t algorithm first s#ggested by Can7ig.
Solved problem
ma%imi7e: ! = 3x1 + 4x2
s.t. 2x1 + x2 6
2x1 + 3x2 "
with: x1 and x2 nonnegative and integral
*ntrod#cing slac) variables x3 and x4 and then solving the res#lting program, with the integer re'#irements ignored, by the simple%
method, we obtain "ablea# 1.
1 . 1/ / . 1 / . 4 4 4
 . 1  . 4  . 4 1 4
/ . / / . 4 1 . 4 4 1
/
1
, . / 1
x
x
x x x x
"ablea# 1
"he first appro%imation is, therefore, x1 = 2*25, x2 = 1*5, which is not integral. "he nonbasic variables are x3 and x4, so the new
constraint is x3 + x4 1. Appending this constraint to "ablea# 1, after the introd#ction of s#rpl#s variable x5 and artificial variable
x6, and solving the res#lting program by two&phase method, we generate "ablea# /.
/ . 1/ / . 4 1 4 4 4
1 1 1 1 4 4
  . 4 1 4 1 4
 . 1 1 . 4 1 4 4 1
.
/
1
 , . / 1
x
x
x
x x x x x
"ablea# /
*t follows from "ablea# / that x1 = 1*5, x2 = 2, x3 = 1, with x4 and x5 nonbasic. (ince this sol#tion is nonintegral, we ta)e x4 + x5
1 as the new constraint. AdEoining this constraint to "ablea# /, after the introd#ction of s#rpl#s variable x6 and artificial variable
x7, and solving the res#lting program by the two&phase method, we generate "ablea# ..
/ . 1/ / . 4 4 1 . 4 4 4 4
1 1 1 1 4 4 4
/ 1 4 / 1 4 4
 . /  . 4 4  . 1 4 1 4
1 . 4 1 . 4 4 1 . 1 4 4 1

.
/
1
2  , . / 1
x
x
x
x
x x x x x x
"ablea# .
Irom "ablea# ., the c#rrent optimal sol#tion is nonintegral, with nonbasic variables x4 and x6. "he new constraint is th#s x4 + x6
1. AdEoining it to "ablea# . and solving the res#lting program by the two&phase method, we obtain x1 = 0, x2 = 3, with ! = 12.
(ince this sol#tion is integral, it is optimal sol#tion to the original integer program.
.4&4
Integer Programming: !"e !ransportation #lgorit"m
A transportation problem involves . so#rces, each of which has available ai i = 1, 2, C, .! #nits of a homogeneo#s prod#ct, and
n destinations, each of which re'#ires bj j = 1, 2, C, n! #nits of this prod#ct. "he n#mbers ai and bj are positive integers. "he cost
cij of transporting one #nit of prod#ct from the ith so#rce to the jth destination is given for each i and j. "he obEective is to develop
an integral transportation sched#le the prod#ct may not be fractionali7ed! that meets all demands from c#rrent inventory at a
minim#m total shipping cost.
*t is ass#med that total s#pply and total demand are e'#al= that is
.
i
n
j
j i
b a
1 1
1!
$'#ation 1! is g#aranteed by creating either a fictitio#s destination with a demand e'#al to the s#rpl#s if total demand is less than
total s#pply, or a fictitio#s so#rce with a s#pply e'#al to the shortage if total demand e%ceeds total s#pply.
Let xij represent the #n)nown! n#mber of #nits to be shipped from so#rce i to destination j. "hen the standard mathematical
model for this problem is:
minimi7e:
.
i
n
j
ij ij
x c !
1 1
s.t.
( ) . i a x
n
j
i ij
, , 1
1
/!
( ) n j b x
.
i
i ij
, , 1
1
'
x x
x x
x x
x x f
A A
A A A
A A
A !
/
/
/
/
is a contin#o#s f#nction, with
'
<
< <
<
x x
x x
x x
x f
A /
A A /
A /
!
"he derivative does not e%ist at A t x , and it is 7ero at x = 4= all three points are in GM,, ,H. $val#ating the obEective f#nction
at each of these points, and at the endpoints , t x , we generate the table
A 4 A 4 A !
, A 4 A ,
x f
x
from which we concl#de that the global ma%im#m on GM,, ,H is ! = A, which is ass#med at the three points , t x and x = 4.
.. :se the three&point interval search to appro%imate the location of the ma%im#m of f=x> = x=5T x> on G4, /4H to within [ = 1.
(ince fII=x> = 52 H 0 everywhere, it follows from "heorem ,! that f=x> is concave, hence #nimodal, on G4, /4H. "herefore, the
three&point interval search is g#aranteed to converge to the global ma%im#m.
First iteration: Cividing G4, /4H into '#arters, we have x1 = , x2 = 14, and x3 = 1 as the three interior points. "herefore
f=x1> = x1=5T x1> = 5=5T 5> = 53*54
f=x2> = x2=5T x2> = 10=5T 10> = 57*08
f=x3> = x3=5T x3> = 15=5T 15> = 10*62
(ince x2 is the interior point generating the greatest val#e of the obEective f#nction, we ta)e the interval G, 1H, centered at x2 as
the new interval of interest.
Second iteration: <e divide G, 1H into '#arters, with x4 = 1., x2 = 14, x5 = 1/. as the three interior points. (o
f=x4> = x4=5T x4> = 7*5=5T 7*5> = 61*56
f=x2> = x2=5T x2> = 10=5T 10> = 57*08 as before!
f=x5> = x5=5T x5> = 12*5=5T 12*5> = 40*10
.2&4
As x4 is the interior point yielding the largest val#e of f=x>, we ta)e the interval G, 14H, centered at x4, as the new interval of
interest.
Third iteration: <e divide G, 14H into '#arters, with x6 = 2./, x4 = 1., and x7 = A.1 as the new interior points. (o
f=x6> = x6=5T x6> = 6*25=5T 6*25> = 5"*11
f=x4> = 61*56 as before!
f=x7> = x7=5T x7> = 8*75=5T 8*75> = 60*88
As x4 yields the largest val#e of f=x>, we ta)e the interval G2./, A.1H. centered at x4, as the new interval of interest.
Fourth iteration: Cividing G2./, A.1H into '#arters, we generate x8 = 2.A1, x4 = 1., and x" = A.1/ as the new interior points.
"h#s
f=x8> = x8=5T x8> = 6*875=5T 6*875> = 60*73
f=x4> = 61*56 as before!
f=x"> = x"=5T x"> = 8*125=5T 8*125> = 61*61
Dow x" is the interior point yielding the largest val#e of the obEective f#nction, so we ta)e the s#binterval centered at x", namely
G1., A.1H, as the new interval for consideration. "he midpoint of this interval however, is within the prescribed tolerance, [ = 1,
of all other point in the interval= hence we ta)e
x = x" = A.1/ with ! = f=x"> = 21.21
,. Kedo Nroblem . #sing the Iibonacci search.
Initial points: "he first Iibonacci n#mber s#ch that OP=1> 20 0 is O8 = 21. <e set P = 8,
3/, . 4
/1
4 /4
P
O
a b
]
is the set of all vectors O s#ch that
( ) ( ) ( ) ( ) ( )
/ / /
/ /
/
1 1
] ] ]
] ]
+ + +
n n
%
x x x x x x
*n geometrical terms, an B&neighborhood aro#nd
]
is the interior and bo#ndary of an n&dimensional sphere of radi#s B centered at
]
.
Gradient Vector and 9essian atrix
"he gradient vector
f
associated with a f#nction f=x1, x2, C, xn> having first partial derivatives is defined by
%
n
x
f
x
f
x
f
f
1
]
1
, , ,
/ 1
"he notation
]
f
signifies the val#e of the gradient at
]
. Ior small displacements from
]
in vario#s directions, the
direction of ma%im#m increase in fO! is the direction of the vector
]
f
.
Example & Ior ( )
.
.
/
/ /
/
1 . / 1
. , , x x x x x x x f , with [ ]
%
. , / , 1
]
,
1
1
1
]
1
/
.
/
/
.
. /
/
1
/ 1
.
/ .
2
x x
x x x
x x
f whence
1
1
1
]
1
1
1
1
]
1
14A
14
1/
! . ! / .
! . ! / / ! 1 .
! / ! 1 2
/ /
. /
f
"herefore, at G1, /, .H
"
, the f#nction increases most rapidly in the direction of G1/, M14, M14AH
"
.
"he Uessian .atrix associated with a f#nction f=x1, x2, C, xn> that has second partial derivatives is
1
1
]
1
j i
f
x x
f
/
i, j = 1, 2,C,n!
"he notation
]
f signifies the val#e of the 6essian matri% at
]
. *n preparation for "heorems , and  below, we shall need
the following:
Definition: An n T n symmetric matri% A one s#ch that A = A
"
! is negative definite negative semi&definite! if O
"
AO is negative
nonpositive! for every n&dimensional vector O ^ 4.
!"eorem &: Let A F GaijH be an n T n symmetric matri%, and define the determinants
.A&4
( ) +
det 1
1
.. /. .1
./ // /1
.1 /1 11
.
// /1
1/ 11
/ 11 1
n
n
'
a a a
a a a
a a a
'
a a
a a
' a '
"hen A is negative definite if and only if '1, '2, C, 'n are all negative= A is negative semi&definite if and only if '1, '2,
C, 'r =r H n> are all negative and the remaining 'Bs are all 7ero.
Example ' Ior the f#nction of $%ample 1,
1
1
1
]
1
.
/
/
/
. /
/
. /
.
. 1
1 /
2 2 4
2 / 2
4 2 2
x x x x
x x x x
x x
f
whence
1
1
1
]
1
1/ 14A 4
14A , 2
4 2 1/
]
f
Ior
]
f , '1 = 1/ \ 4, so that f
]
.
1esults $rom Calculus
!"eorem ': *f fO! is contin#o#s on a closed and bo#nded region, then fO! has a global ma%im#m and also a global minim#m!
on that region.
!"eorem .: *f fO! has a local ma%im#m or a local minim#m! at O
L
and if
f
e%ists on some B&neighborhood aro#nd O
L
, then
. 4
f
!"eorem /: *f fO! has second partial derivatives on an B&neighborhood aro#nd O
L
, and if
4
f
and
f is negative
definite, then fO! has a local ma%im#m at O
L
.
*t follows from "heorem / and . that a contin#o#s fO! ass#mes its global ma%im#m among those points at which
f
does not
e%ist or among those points at which
4 f
stationary points! M #nless the f#nction ass#mes even larger val#es as O
"
O _ W.
*n the latter case, no global ma%im#m e%ists.
Analytical sol#tions based on calc#l#s are even harder to obtain for m#ltivariable programs than for single&variable programs, and
so, once again, n#merical methods are #sed to appro%imate local! ma%ima to within prescribed tolerance.
!"e et"od o$ Steepest #scent
@hoose an initial vector O4, ma)ing #se of any prior information abo#t where the desired global ma%im#m might be fo#nd. "hen
determine vectors O1, O/, O., J by the iterative relation
k
f
k k k
+
+
1 /!
6ere
k
is a positive scalar which ma%imi7es
( )
k
f f
k
+
= this single&variable program is solved by the methods of
single&variable optimi7ation. *t is best if
k
represents a global ma%im#m= however, a local ma%im#m will do. "he iterative
process terminates if and when the differences between the val#es of the obEective f#nction at two s#ccessive O&vectors is smaller
than a prescribed tolerance. "he last&comp#ted O&vector becomes the final appro%imation to OL.
!"e Ne%ton51ap"son et"od
@hoose an initial vector O4, as in the method of steepest ascent. +ectors O1, O/, O., J then are determined iteratively by
k
k
f
f k k
,
_
1
1
.!
"he stopping r#le is the same as for the method of steepest ascent.
"he Dewton&Kaphson method will converge to a local ma%im#m if f
is negative definite,
1
f
e%ists and is negative definite.
*f O4 is not chosen correctly, the method may converge to a local minim#m or it may not converge at all. *n either case, the
iterative process is terminated and then beg#n anew with a better initial appro%imation.
!"e Fletc"er5Po%ell et"od
"his method, an eight&step algorithm, is beg#n by choosing an initial vector
]
and prescribing a tolerance B, and by setting an n
T n matri% > e'#al to the identity matri%. Qoth
]
and > are contin#ally #pdated #ntil s#ccessive val#es of the obEective f#nction
differ by less than B, where#pon the last val#e of
]
is ta)en as O
L
.
(tep1: $val#ate !
]
f and
.
]
f (
(tep/: Cetermine
s#ch that ( ) + >
]
f is ma%imi7ed when .
(et . > C
(tep.: Cesignate
C
]
+
as the #pdated val#e of .
]
(tep,: @alc#late !
]
f for the #pdated val#es of .
]
*f
, <
go to (tep= if not, go to (tep2.
(tep: (et , ! ,
]
f and stop.
(tep2: $val#ate @ =
]
f
for the #pdated vector ,
]
and set ` = Q M @.
.3&4
(tep1: @alc#late the n T n matrices
> >``
>` `
1 &
; and CC
` C
1
L
"
"
,
_
,
_
%
%
(tepA: Cesignate > 9 L 9 ; as the #pdated val#e of >. (et
is
negative semi&definite for all O in R.
!"eorem ;: *f fO! is concave on R, then any local ma%im#m on R is a global ma%im#m on R.
"hese two theorems imply that, if f
is negative definite everywhere, then fO! is strictly concave everywhere!, and the sol#tion to program 1! is
#ni'#e.
Solved Problems
1. ;a%imi7e: ! . ! 1
/
. . / 1
+ x x x x !
6ere !. . ! 1 ! , ,
/
. . / 1 . / 1
+ x x x x x x x f "he gradient vector,
[ ] , . . , , 1
/
. 1 /
%
x x x f
e%ists everywhere
and is 7ero only at
O1 = G4, 1, 1H
%
and O/ = G4, 1, M 1H
%
<e have fO1! = M / and fO/! = /. Q#t f=x1, x/, x.! becomes arbitrarily large as x. for instance! does so= hence no global ma%im#m
e%ists. "he vector O/ is not even the site of a local ma%im#m= rather, it is a saddle point, as is O1.
/. ;inimi7e:
( ) ( ) . 14 
/
/
/
1
+ + x x !
;#ltiplying this obEective f#nction by M 1, we obtain the e'#ivalent ma%imi7ation program
,4&4
;a%imi7e:
( ) ( ) 14 
/
/
/
1
x x !
for which
[ ] . ,  /
/ 1
%
x x !
"h#s there is a single stationary point, , , 
/ 1
x x at which ! = M 14. Dow, as
,
/
/
/
1
+ x x ! becomes arbitrarily small= conse'#ently, !
L
= M 14 is the global ma%im#m, and !
L
= 9 14 is the global minim#m
for the original minimi7ation program. "he minim#m is, of co#rse, also ass#med as . 1,12 . . , /.21 . / 
/ 1
x x
.. :se the method of steepest ascent to
minimi7e:
( ) ( ) 14 
/
/
/
1
+ + x x !
>oing over to the e'#ivalent program
ma%imi7e:
( ) ( ) 14 
/
/
/
1
x x !
1!
we re'#ire a starting program sol#tion, which we obtain by a random sampling of the obEective f#nction over the region M 14 x1,
x2 14. "he sample points and corresponding !&val#es are shown in the table below. "he ma%im#m !&entry is M .2..A, occ#rring
at O4 = G2.21, .A31H
%
, which we ta)e as the initial appro%imation to O
L
. "he gradient of the obEective f#nction for program 1! is
( )
( )
1
1
]
1
/
1
/
 /
x
x
f
x1 M A..1 M 4.313A 3./41 3./4 2.31 A.,11 A./4/ M 3.11. M 3...1 M .13,
x2 M 1.433 M A.44 M /./, 1.,2 .A31 M 3.3, M .143 M 2.31, A.12. M 4.4/14
! M 1,,.4 M 1,,./ M 34.21 M 1A.3 M .2.A M /13., M 1/..3 M /,1.. M 123./ M A,.,A
First iteration.
( )
( )
1
]
1
1
1
]
1
+
1
]
1
,33 .  A31 . 
1// . A 31 . 2
A31 .  /
 31 . 2 /
A31 . 
31 . 2
4
4
f
( ) ( ) ( )
3 . .2 . . 142 . . 142
14 ,33 .  A31 .   1// . A 31 . 2
/
/
/
4
4
+
+
f f
:sing the analytical methods described in previo#s chapter, we determine that this f#nction of ass#mes a global! ma%im#m at
.  . 4
4
"h#s,
( )
( )
1
]
1
1
]
1
1,/ . .
/.2 . /
 . 4 ,33 .  A31 . 
 . 4 1// . A 31 . 2
4
4 4 1
f
with fO1! = M 14.44. (ince the difference between fO4! = M .2.A and fO1! = M 14.44 is significant, we contin#e iterating.
Second iteration.
( )
( )
1
]
1
1
1
]
1
+
1
]
1
444A . 4 1,/ . .
4441 . 4 /.2 . /
1,/ . . /
 /.2 . / /
1,/ . .
/.2 . /
1
1
f
( ) ( ) ( )
( )
1 A /
/
/
1
14 14 .A/ . 2 44 . 2
14 444A . 4 1,/ . .  4441 . 4 /.2 . /
1
+
+
f f
:sing the analytical methods described in previo#s chapter, we find that this f#nction of has a global! ma%im#m at
. ,343 . 4
1
"h#s,
( )
( )
1
]
1
1
]
1
+
+
1,/ . .
/.2 . /
,343 . 4 444A . 4 1,/ . .
,343 . 4 4441 . 4 /.2 . /
1
1 1 /
f
(ince O1 = O/ to fo#r significant fig#res!, we accept O
L
= G/./.2, ..1,/H
%
, with !
L
= M 14.44, as the sol#tion to program 1!. "he
sol#tion to the original minimi7ation program is then O
L
= G/./.2, ..1,/H
%
, with !
L
= 9 14.44. @ompare this with the res#lts in
Nroblem /.
,. :se the Dewton&Kaphson method to
ma%imi7e:
( ) ( ) 14 
/
/
/
1
+ + x x !
to within a tolerance of 4.4
Irom Nroblem . we ta)e the initial appro%imation O4 = G2.21, .A31H
%
, with fO4! = M .2.A. "he gradient vector, 6essian matri%,
and inverse 6essian matri% for this obEective f#nction are, respectively,
,1&4
( )
( )
1
1
]
1
/
1
/
 /
x
x
f
1
]
1
/ 4
4 /
f 1
]
1
 . 4 4
4  . 4
1
f
for all x1 and x2.
First iteration.
( )
( )
1
]
1
1
1
]
1
,33 . 
1// . A
A31 .  /
 31 . 2 /
4
f
4
4
1
4 1
,
_
f
f
1
]
1
1
]
1
1
]
1
1
]
1
1,/ . .
/.2 . /
,33 . 
1// . A
 . 4 4
4  . 4
A31 . 
31 . 2
with fO1! = M 14.44. (ince
fO1! M fO4! = M 14.44 M M .2.A! = /2.A \ 4.4
we contin#e iterating.
Second iteration.
( )
( )
1
]
1
1
1
]
1
444A . 4
4441 . 4
1,/ . . /
 /.2 . / /
1
f
1
1
1
1 /
,
_
f
f
1
]
1
1
]
1
1
]
1
1
]
1
1,/ . .
/.2 . /
444A . 4
4441 . 4
 . 4 4
4  . 4
1,/ . .
/.2 . /
with fO/! = M 14.44. (ince fO/! M fO1! = 4 P 4.4, we ta)e O
L
= O/ = G/./.2, ..1,/H
%
, with !
L
= fO/! = M 14.44.
. A maEor oil company wants to b#ild a refinery that will be s#pplied from three port cities. Nort Q is located .44 )m east and ,44
)m north of Nort A, while Nort @ is ,44 )m east and 144 )m so#th of Nort Q. Cetermine the location of the refinery so that the
total amo#nt of pipe re'#ired to connect the refinery to the ports is minimi7ed.
"he obEective is tantamo#nt to minimi7ing the s#m of the distances between the refinery and the three ports. As an aid to
calc#lating this s#m, we establish a coordinate system, with Nort A as the origin. *n this system, Nort Q has coordinates .44, ,44!
and Nort @ has coordinates 144, .44!. <ith x1, x2! designating the #n)nown coordinates of the refinery, the obEective is
minimi7e:
( ) ( ) ( ) ( )
/
/
/
1
/
/
/
1
/
/
/
1
.44 144 ,44 .44 + + + + + x x x x x x !
(olve the problem to within 4./ )m by the Iletcher&Nowell method.
"his problem is e'#ivalent to a ma%imi7ation program with obEective f#nction
( ) ( ) ( ) ( ) ( )
/
/
/
1
/
/
/
1
/
/
/
1
.44 144 ,44 .44 + + + x x x x x x f
1!
and gradient vector
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) 1
1
1
1
1
]
1
/
/
/
1
/
/
/
/
1
/
/
/
/
1
/
/
/
/
1
1
/
/
/
1
1
/
/
/
1
1
.44 144
.44
,44 .44
,44
.44 144
144
,44 .44
.44
x x
x
x x
x
x x
x
x x
x
x x
x
x x
x
f /!
"o initiali7e the Iletcher&Nowell method, we set B = 4./ and
1
]
1
1 4
4 1
>
and choose [ ] , /44 , ,44
]
%
which appears to be a good appro%imation to the optimal location of the refinery.
4te31:
( ) ( ) /44 , ,44
]
f f
( ) ( ) ( ) ( ) ( ) ( ) 4 . 3A1 144 .44 /44 144 /44 ,44
/ / / / / /
+ + +
1
]
1
12.,, . 4
.3/32 . 4
]
f
4te32:
( )
,
_
1
]
1
,
_
1
]
1
1
]
1
+
1
]
1
12.,, . 4 /44
.3/32 . 4 ,44
12.,, . 4
.3/32 . 4
1 4
4 1
/44
,44
>
]
f f f
,/&4
( ) ( )
( ) ( )
( ) ( )
/ /
/ /
/ /
12.,, . 4 144 .3/32 . 4 .44
12.,, . 4 /44 .3/32 . 4 144
12.,, . 4 /44 .3/32 . 4 ,44
+ +
+ +
+ +
;a)ing a three&point interval search of G4, ,/H, we determine V
L
d /1/.. "herefore,
( )
1
]
1
1
]
1
1
]
1
/. . 12/
4, . A.
132,, . 4
.3/32 . 4
1 4
4 1
 . /1/ > C
4te33:
1
]
1
1
]
1
+
1
]
1
+
/. . .2/
4 . .12
/. . 12/
4, . A.
/44
,44
C
]
which we ta)e as the #pdated [ ] . /. . .2/ , 4 . .12
]
:
]
%
4te34:
( ) ( ) 12 . 314 /. . .2/ , 4 . .12
]
f f
( ) / . 4 /3 . 12 4 . 3A1 12 . 314 >
4te36:
1
]
1
44.13, . 4
411/41 . 4
@
]
f
1
]
1
1
]
1
1
]
1
124/A . 4
./11 . 4
44.13, . 4
4411/41 . 4
1.2,, . 4
.3/32 . 4
@
4te37:
[ ]
[ ]
[ ]
[ ]
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
1
]
1
A,A11 . 4 .A3/ . 4
.A3/ . 4 11A3 . 4
1A4. . 4 /,,2/ . 4
/,,2/ . 4 14./ . 4
2A1 . 4
1
1 4
4 1
124/A . 4 , ./11 . 4
124/A . 4
./11 . 4
1 4
4 1
2A1 . 4
1
> >
2A1 . 4
1
2A1 . 4
124/A . 4
./11 . 4
1 4
4 1
124/A . 4 , ./11 . 4 >
/1 . 11 1A1 . 34
1A1 . 34 ,/1 . ,2
/2.13 1.,1
1.,1 3 . 231/
/1 . 14
1
/. . 12/ , 4, . A.
/. . 12/
4, . A.
/1 . 14
1
CC
/1 . 14
1
L
/1 . 14
124/A . 4
./11 . 4
/. . 12/ , 4, . A. C
%
%
%
%
4te38:
1
]
1
1
]
1
+
1
]
1
+
1
]
1
+ +
.2 . 11 A/A . A3
A/A . A3 /23 . ,1
A,A11 . 4 .A3/ . 4
.A3/ . 4 11A3 . 4
/1 . 11 1A1 . 34
1A1 . 34 ,/1 . ,2
1 4
4 1
L >
which we ta)e as the #pdated >. <e also #pdate S = M 314.12 and
1
]
1
44.13, . 4
411/41 . 4
4te32:
( )
( ) ( )
( ) ( )
( ) ( )
/ /
/ /
/ /
34, . 2 /. . 2/ 2,31 . . 4 . .A.
34, . 2 11 . .1 2,31 . . 4 . 12
34, . 2 /. . .2/ 2,31 . . 4 . .12
34, . 2 /. . .2/
2,31 . . 4 . .12
44.13, . 4
411/41 . 4
.2 . 11 A/A . A3
A/A . A3 /23 . ,1
/. . .2/
4 . .12
>
]
+ +
+ +
+ +
,
_
1
]
1
,
_
1
]
1
1
]
1
+
1
]
1
+
f
f f
;a)ing a three&point interval search of G4, 14H, we determine V
L
d 1./. "herefore,
,.&4
( )
1
]
1
1
]
1
1
]
1
2AA4 . A
2/1 . ,
44.13, . 4
411/41 . 4
.2 . 11 A/A . A3
A/A . A3 /23 . ,1
/ . 1 > C
4te33:
1
]
1
1
]
1
+
1
]
1
+
3/ . .14
3, . .11
2AA4 . A
2/1 . ,
/. . .2/
4 . .12
C
]
which we ta)e as the #pdated .
]
4te34:
( ) ( )
( ) / . 4 1A . 4 12 . 31 A . 314
A . 314 3/ . .14 , 3, . .11
]
<
f f
4te35:
1
]
1
3/ . .14
31 . .11
]
and ( ) A . 314
f
"h#s the problem is solved by )m. A . 314 with )m, 3/ . .14 )m, 31 . .11
/ 1
+
! x x
Nonlinear Programming: ultivariable 6ptimi7ation %it" Constraints
Standard Forms
<ith O F Gx1, x2, C, xnH
%
, standard form for nonlinear programs containing only e'#ality constraints is
ma%imi7e: ! = fO!
s.t. E1O! = 4
E2O! = 4 1!
JJJ..
E.O! = 4
with: . P n fewer constraints than variables!
As in previo#s chapter minimi7ation programs are converted into ma%imi7ation programs by m#ltiplying the obEective f#nction by
M 1.
(tandard form for nonlinear programs containing only ine'#ality constraints is either
ma%imi7e: ! = fO!
s.t. E1O! 4
E2O! 4 /!
JJJ..
E.O! 4
or
ma%imi7e: ! = fO!
E1O! 4
E2O! 4 .!
JJJ..
E.O! 4
with: O 4
"he two forms are e'#ivalent: /! goes over into .! with . = 3! #nder the s#bstit#tion O = : M +, with : 4 and + 4= on the
other hand, .! is E#st /! in the special case 3 = . + n and E. 9 iO! = M xi i = 1, /, J, n!. Iorm .! is appropriate when the
sol#tion proced#re re'#ires nonnegative variables. *n 1!, /!, or .!, f is a nonlinear f#nction, b#t some or all of the EBs may be
linear.
Donlinear programs not in standard form are solved either by p#tting them in s#ch form or by s#itably modifying the sol#tion
proced#re given below for programs in standard form.
)agrange ultipliers
"o solve program 1!, first form the QaEranEian f2nctin
( ) ( ) ( )
.
i
i i . n
E f x x x Q
1
/ 1 / 1
, , , , , , , ,!
where Vi i = 1, /, J, .! are #n)nown! constants called QaEranEe .2lti3liers. "hen solve the system of n + . e'#ations
( )
( ) . i
Q
n j
x
Q
i
j
, , / , 1 4
, , / , 1 4
!
"heorem 1: *f a sol#tion to program 1! e%ists, it is contained among the sol#tions to system !, provided fO! and EiO! i = 1, /,
J, .! all have contin#o#s first partial derivatives and the . T n Wacbian .atrix,
1
1
]
1
j
i
x
E
c
has ran) . at O = O
L
.
,,&4
"he method of Lagrange m#ltipliers is e'#ivalent to #sing the constraint e'#ations to eliminate certain of the x&variables from the
obEective f#nction and then solving an #nconstrained ma%imi7ation problem in the remaining x&variables.
Ne%ton51ap"son et"od
(ince ( ) ( ) Q x x x Q
. n
, , , , , , ,
/ 1 / 1
is nonlinear, it is #s#ally impossible to solve ! analytically. 6owever, since
the sol#tions to ! are the stationary points of Qe!, and since "heorem . of previo#s chapter! the ma%ima and minima of Qe!
occ#r among these stationary points, it sho#ld be possible to #se the Dewton&Kaphson method see previo#s chapter! to
appro%imate the XrightY e%trem#m of Qe!= that is, the one that corresponds to the optimal sol#tion of 1!. "he iterative form#la
applicable here is
k
k
f
f k k
,
_
1
1
2!
"his approach is of limited val#e beca#se, as in previo#s chapter, it is very diffic#lt to determine an ade'#ate e4. Ior an incorrect
e4, the Dewton&Kaphson method may diverge or may converge to the XwrongY e%trem#m of Qe!. *t is also possible for the
method to converge when no optimal sol#tion e%ists.
Penalty Functions
An alternative approach to solving program 1! involves the #nconstrained program
ma%imi7e: ( ) ( )
.
i
i i
E 3 f !
1
/
]
1!
where 3i \ 4 are constants still to be chosen! called 3enalt0 +eiE,ts. "he sol#tion to program 1! in the sol#tion to program 1!
when each EiO! = 4. Ior large val#es of the 3i, the sol#tion to 1! will have each EiO! near 7ero to avoid adverse effects on the
obEective f#nction from the terms ( )=
/
i i
E 3 and as each ,
i
3 each ( ) . 4
i
E
*n practice, this process cannot be accomplished analytically e%cept in rare cases. *nstead, program 1! is solved repeatedly by the
modified pattern search described in previo#s chapter, each time with either a new set of increased penalty weights or a decreased
step si7e. $ach pattern search with a specified set of penalty weights and a given step si7e is one phase of the sol#tion proced#re.
"he starting vector for a partic#lar phase is the final vector from the phase immediately preceding it. Nenalty weights for the first
phase are chosen small, often 154 = 4.4/= the first step si7e generally is ta)en as 1.
@onvergence of this proced#re is affected by the rates at which the penalty weights are increased and the step si7e is decreased.
Cecisions governing these rates are more a matter of art than of science.
2u"n5!ucker Conditions
"o solve program .!, first rewrite the nonnegativity conditions as M x1 4, M x2 4, J, M xn 4, so that the constraint set is . + n
ine'#ality re'#irements each with a less than or e'#als sign. De%t add slac) variables , , , ,
/
/
/
/
/
1 . n n n
x x x
+ + +
respectively, to the left&hand sides of the constraints, thereby converting each ine'#ality into an e'#ality. 6ere the slac) variables
are added as s'#ared terms to g#arantee their nonegativity. "hen form the Lagrangian f#nction
( ) ( ) [ ] [ ]
+ +
+
+
+ +
.
i
n .
. i
i n i i i n i i
x x x E f Q
1 1
/ /
A!
where
n .+
, , ,
/ 1
are Lagrange m#ltipliers. Iinally solve the system
( ) . n j
x
Q
j
+
/ , /, 1, 4
3!
( ) . n i
Q
i
+
/ , /, 1, 4
14!
( ) . n j
i
+ / , /, 1, 4 11!
$'#ations 3! thro#gh 11! constit#te the D2,n5%2cker cn1itins for program /! and .!."he first two sets, 3! and 14!, follow
directly from Lagrange m#ltiplier theory= set 11! is )nown as the constraint '#alification. Among the sol#tions to the S#hn&
"#c)er conditions will be the sol#tion to program .! if fO! and each EiO! have contin#o#s first partial derivatives.
et"od o$ Feasible (irections
"his is a five&step algorithm for solving program /!. "he method is applicable only when the feasible region has an interior, and
then it will converge to the global ma%im#m only if the initial appro%imation is XnearY the sol#tion. "he feasible region will have
no interior if two of the ine'#ality constraints have arisen from the conversion of an e'#ality constraint.
(tep1: Cetermine an initial, feasible appro%imation to the sol#tion, designating it Q.
(tep/: (olve the following linear program for the 11, 12, C, 1n 9 1:
ma%imi7e: ! = 1n + 1
s.t.
,&4
( )
( )
( ) +
+ +
+ +
+ +
+
+
+
+
3 n 3 n
n
3 3 3
n n
n
n n
n
n
n
E 1 k 1
x
E
1
x
E
1
x
E
E 1 k 1
x
E
1
x
E
1
x
E
E 1 k 1
x
E
1
x
E
1
x
E
1
x
f
1
x
f
1
x
f
1 /
/
1
1
/ 1 /
/
/
/
/
1
1
/
1 1 1
1
/
/
1
1
1
1
1 /
/
1
1
4
1/!
with: 1j 1 j = 1, /, J, n 9 1!
6ere ki i = 1, /, J, 3! is 4 if EiO! is linear and 1 if EiO! is nonlinear.
(tep.: *f 1n + 1 = 4, then O
L
= Q= if not, go to (tep ,.
(tep,: (et C = G11, 12, C, 1nH
%
. Cetermine a nonnegative val#e for V that ma%imi7es fQ 9 VC! while )eeping Q 9 VC feasible=
designate this val#e as V
L
.
(tep: (et Q = Q 9 V
L
C and ret#rn to (tep/.
(olved Nroblems
1. ma%imi7e: ! = /x1 +x1 x/ + .x.
s.t. x1
/
9 x/ = .
*t is apparent that for any large negative x1 there is a large negative x/ s#ch that the constraint e'#ation is satisfied. Q#t then
! d x1x/ _ W. "here is no global ma%im#m.
/. minimi7e: ! = x1 9 x/ 9 x.
s.t. x1
/
9 x/ = .
x1 9 .x/ 9 /x. = 1
"he given program is e'#ivalent to the #nconstrained minimi7ation of
7 = f x1
/
9 x1 9 ,!
which obvio#sly has a sol#tion. <e may therefore apply the method of Lagrange m#ltipliers to the original program standardi7ed
as
ma%imi7e ! = M x1 M x/ M x.
s.t. x1
/
9 x/ M . = 4 1!
x1 9 .x/ 9 /x. M 1 = 4
6ere, fx1, x/, x.! = M x1 M x/ M x., n = . variables!, . = / constraints!,
E1x1, x/, x.! = x1
/
9 x/ M . E/x1, x/, x.! = x1 9 .x/ 9 /x. M 1
"he Lagrangian f#nction is then
Q = M x1 M x/ M x.! M V1x1
/
9 x/ M .! M V/x1 9 .x/ 9 /x. M 1!
and system ! becomes
( )
( )
( )
( ) ( )
( ) ( ) 2 4 1 / .
 4 .
, 4 / 1
. 4 . 1
/ 4 / 1
. / 1
/
/
/
1
1
/
.
/ 1
/
/ 1 1
1
+ +
x x x
Q
x x
Q
x
Q
x
Q
x
x
Q
(#ccessively solving ,! for V/, .! for V1, /! for x1, ! for x/, and 2! for x., we obtain the #ni'#e sol#tion V/ = M 4., V1 = 4., x1 =
M 4., x/ = /.1, and x. = M 4..1, with
! = M x1 M x/ M x. = M M 4.! M /.1 M M 4..1! = M 1.A1
(ince the first partial derivatives of fx1, x/, x.!, E1x1, x/, x.!, and E/x1, x/, x.! are all contin#o#s, and since
1
]
1
1
1
1
1
]
1
/ . 1
4 1 /
c
1
.
/
/
/
1
/
.
1
/
1
1
1
x
x
E
x
E
x
E
x
E
x
E
x
E
is of ran) / everywhere the last two col#mns are linearly independent everywhere!, either x1 = M 4., x/ = /.1, x. = M 4..1 is the
optimal sol#tion to program 1! or no optimal sol#tion e%ists. @hec)ing feasible points in the region aro#nd M 4., /.1, M 4..1!,
,2&4
we find that this point is indeed the location of a global! ma%im#m for program 1!. "herefore, it is also the location of a global
minim#m for the original program, with !
L
= M M 1.A1! = 1.A1.
.. Cisregarding Nroblem 1 #se the Dewton&Kaphson method to
ma%imi7e: ! = /x1 +x1 x/ + .x/
s.t. x1
/
9 x/ = .
6ere, Q = /x1 9 x1x/ 9 .x/! M V1x1
/
9 x/ M .!. "herefore,
1
1
1
]
1
+
+
+
.
.
/ /
/
/
1
1 1
1 1 /
x x
x
x x
Q
1
1
1
]
1
4 1 /
1 4 1
/ 1 /
1
1 1
x
x
Q
1
.
1
4
X
Q
1
1
1
]
1
4 1 /
1 4 1
/ 1 /
4
X
Q
( )
1
1
1
]
1
1 , 1
, , /
1 / 1
2
1 1
4
X
Q
and
( ) [ ]
%
Q
Q . 14 , . 14 , . 1
4 4
1
4 1
4ecn1 iteratin
1
1
1
]
1
3 ,
4
3 /A
1
X
Q
1
1
1
]
1
4 . /
. 4 .
/ . /4
.
1
1
X
Q
( )
1
1
1
]
1
3 22 3
22 , 2
3 2 3
1/
1 1
1
X
Q
and
( ) [ ]
%
Q
Q . 11 , . A , . /
1 1
1
1 /
+
+
3
3
3
3
x x
(ince the 6essian matri%
1
]
1
1 1
1 1
]
/ / 2
2 1A 2
3 3
3 3
!
is negative definite for every positive val#e of 31, !] is a strictly concave f#nction, and its sole stationary point m#st be a global
ma%im#m. "herefore, letting 31 _ 9W we obtain the optimal sol#tion to the original program:
1 1
,

x x
/ /
,

x x
with
( ) ( ) / . , 1 1 . ,
/
/
/
1
x x !
. :se the penalty f#nction approach to
minimi7e: ! = x1 M x/!
/
9 x. M 1!
/
91
s.t. x1

9 x/

9 x.

= 12
N#tting this program in standard form, we have
,1&4
ma%imi7e: ! = M x1 M x/!
/
M x. M 1!
/
M 1 1!
s.t. x1

9 x/

9 x.

M 12 = 4
Ior program 1!, penalty f#nction becomes
ma%imi7e:
( ) ( ) ( )
/

.

/

1 1
/
.
/
/ 1
12 1 1 ] + + x x x 3 x x x !
/!
6,ase 1. <e set 31 = 4.4/ in /! and consider the program
ma%imi7e:
( ) ( ) ( )
/

.

/

1
/
.
/
/ 1
12 4/ . 4 1 1 ] + + x x x x x x !
.!
Arbitrarily selecting G4, 4, 4H
%
as o#r initial vector, and setting , = 1, we apply the modified pattern search to program .!. "he
res#lt after 1A f#nctional eval#ations is G1, 1, 1H
%
, with
f1, 1, 1! = M 1 and E11, 1, 1! = M 1.
6,ase 2. (ince E11, 1, 1! = M 1. ^ 4, the constraint in program 1! is not satisfied. "o improve this sit#ation, we increase 31 in /!
to 4./ and consider the program
ma%imi7e:
( ) ( ) ( )
/

.

/

1
/
.
/
/ 1
12 / . 4 1 1 ] + + x x x x x x !
,!
"a)ing G1, 1, 1H
%
from 6,ase1 as the initial appro%imation, we apply the modified pattern search to ,!, still )eeping , = 1. "he
res#lt remains G1, 1, 1H
%
, indicating that the constraint cannot be satisfied in integers.
6,ase 3. (ince increasing 31 did not improve the c#rrent sol#tion, we ret#rn to program .!, red#ce , to 4.1, and ma)e a new
pattern search, again with G1, 1, 1H
%
as initial appro%imation. "he res#lt is G1., 1., 1H
%
with
f1., 1., 1! = M 1 and E11., 1., 1! = 4.1A1
@ontin#ing in this manner, we complete the table below. :sing the res#lts of 6,ase 3, we concl#de that x1
L
= 1.,32, x/
L
= 1.,32,
%.
L
= 1.44., with !
L
= 91.444, appro%imates the optimal sol#tion to the original minimi7ation program.
Nhase 31 , Iinal +ecgtor O fO! E1O!
x1 x/ x.
1 4.4/ 1 1 1 1 M 1 M 1.
/ 4./ 1 1 1 1 M 1 M 1.
. 4.4/ 4.1 1. 1. 1 M 1 4.1A1
, 4./ 4.1 1. 1. 1 M 1 4.1A1
 4.4/ 4.41 1.,3 1. 1 M 1.444 M 4.42/.
2 4./ 4.41 1.,3 1. 1.41 M 1.444 M 4.411.
1 4./ 4.441 1.132 1.,32 1.44/ M 1.444 M 4.44.3
A / 4.441 1.,32 1.,32 1.44. M 1.444 4.441/
3 /4 4.441 1.,32 1.,32 1.44. M 1.444 4.441/
2. (olve the following program by #se of the S#hn&"#c)er conditions:
minimi7e:
. / 1 . / . 1 / 1
/
.
/
/
/
1
 14 / 1/ 2 , 14  x x x x x x x x x x x x ! + + + + +
s.t. x1 9 /x/ 9 x. ,
with: all variables nonnegative
Iirst transforming into the standard form and then introd#cing s'#ared slac) variables, we obtain
ma%imi7e:
. / 1 . / . 1 / 1
/
.
/
/
/
1
 14 / 1/ 2 , 14  x x x x x x x x x x x x ! + + +
s.t. M x1 M /x/ M x. 9 , 9 x,
/
= 4
M x1 9 x
/
= 4
M x/ 9 x2
/
= 4
M x. 9 x1
/
= 4
Ior this program, the Lagrangian f#nction is
( ) ( ) ( ) ( )
/
1 . ,
/
2 / .
/
 1 /
/
, . / 1 1
. / 1 . / . 1 / 1
/
.
/
/
/
1
, /
 14 / 1/ 2 , 14 
x x x x x x x x x x
x x x x x x x x x x x x Q
+ + + + +
+ + +
"a)ing the derivatives indicated in 3! and 14!, we have
,A&4
11! 4
14! 4
3! 4
A! 4 , /
1! 4 /
2! 4 /
! 4 /
,! 4 /
.! 4  1/ 2 /4
/! 4 / 14 1/ , 14
1! 4 / 2 , /
/
1 .
,
/
2 /
.
/
 1
/
/
, . / 1
1
1 ,
1
2 .
2
 /

, 1
,
, 1 / 1 .
.
. 1 . 1 /
/
/ 1 . / 1
1
+ +
+ + +
+ + + +
+ + + +
x x
Q
x x
Q
x x
Q
x x x x
Q
x
x
Q
x
x
Q
x
x
Q
x
x
Q
x x x
x
Q
x x x
x
Q
x x x
x
Q
"hese e'#ations can be simplified. (et
s1 = x,
/
1/!
$'#ations ,! thro#gh 1! imply respectively that either V1 or x,, either V/ or x, either V. or x2, either V, or x1, e'#als 7ero. Q#t, by
3! thro#gh 1/!, x,, x, x2, and x1 are 7ero if and only if s1, x1, x/, and x, are respectively 7ero. "h#s, ,! thro#gh 1! and 3! thro#gh
1/! are e'#ivalent to the system
4
4
4
4
. ,
/ .
1 /
1 1
x
x
x
s
1.!
"here are 12 sol#tions to this system. Kes#lts are listed in the table below
V1 V2 V3 V4 x1 x2 x3 s1
1 4 4 4 4 11. &. &. &,
/ 4 4 4 11 & &. 4 &1
. 4 4 2 4 11. 4 &. &,
, 4 4 2 11 1 4 4 &.
 4 &1.2,. 4 4 4 &,.2,. &..4.2 &12../
2 4 / 4 11 4 &1 4 &/
1 4 &.. 1. 4 4 4 &4./ &,./
A 4 &/ 14  4 4 4 &,
3
4..A4
3 4 4 4
1,..
2 &/./.A &.AA1 4
1
4 1.12, 4 4
1,.
.
/.3,
1
4./3
, 4 4
1
1 &../ 4 1A.A 4 2.. 4 &/.. 4
1
/ 2 4 &A 11 , 4 4 4
1
. 2.2/. &A.1.A 4 4 4 1.41
4.3A
 4
1
, 1 &/ 4 &., 4 / 4 4
1
 A &2. &/4A 4 4 4 , 4
,3&4
1
2 J J J J 4 4 4 4
"he only row in this table having nonnegative entries for all variables, as re'#ired by the S#hn&"#c)er conditions, is row 14 with
!
L
= ../..
4&4