You are on page 1of 50

Nonnegativity Conditions

Any variable not already constrained to be nonnegative is replaced by the difference of two new variables which are so
constrained.
Linear constraints are of the form:
i
n
j
j ij
b x a ~
1

where ~ stands for one of the relations , , = not necessarily the same one for each i!. "he constants bi may always be ass#med
nonnegative.
$%ample: "he constraint 2x1 3x2 + 4x3 5 is m#ltiplied by 1 to obtain
2x1 + 3x2 4x3 5, which has a nonnegative right&hand side.
Slack Variables and Surplus Variables
A linear constraint of the form


i j ij
b x a
can be converted into an e'#ality by adding a new, nonnegative variable to the
left&hand side of the ine'#ality. (#ch a variable is n#merically e'#al to the difference between the right& and left&hand sides of the
ine'#ality and is )nown as a slack variable.
Example: A man#fact#rer is beginning the last wee) of prod#ction of fo#r different models of wooden television consoles,
labeled *, **, ***, and *+, each of which m#st be assembled and then decorated. "he models re'#ire ,, -, ., and - h, respectively,
for assembling and /, 1.-, ., and . h, respectively, for decorating. "he profits on the models are 01, 0,, 02, and 03, respectively.
"he man#fact#rer has .4 444 h available for assembling these prod#cts 1-4 assemblers wor)ing ,4 h5w)! and /4 444 h available
for decorating -44 decorators wor)ing ,4 h5w)!. 6ow many of each model sho#ld the man#fact#rer prod#ce d#ring this last
wee) to ma%imi7e profit8 Ass#me that all #nits made can be sold.
"he first constraint in above problem is
4x1 + 5x2 + 3x3 + 5x4 30 000
"his ine'#ality is transformed into the e'#ation
4x1 + 5x2 + 3x3 + 5x4 9 x5 = 30 000
by adding the slac) variable x5 to the left&hand side of the ine'#ality. 6ere x5 represents the n#mber of assembly ho#rs available to
the man#fact#rer b#t not #sed.
A linear constraint of the form


i j ij
b x a
can be converted into an e'#ality by s#btracting a new, nonnegative variable from
the left&hand side of the ine'#ality. (#ch a variable is n#merically e'#al to the difference between the left& and right&hand sides of
the ine'#ality and is )now as a s#rpl#s variable. *t represents e%cess inp#t into that phase of the system modeled by the constraint.
Example: :niversal ;ines *nc. operates three mines in <est +irginia. "he ore from each mine is separated into two grades
before it is shipped= the daily prod#ction capacities of the mines, as well as their daily operating costs, are as follows:
6igh&>rade ?re,
tons5day
Low&>rade ?re,
tons5day
?perating @ost, 014445
day
;ine * , , /4
;ine ** 2 , //
;ine *** 1 2 1A
:niversal has committed itself to deliver -, tons of high&grade ore and 2- tons of low&grade ore by the end of the wee). *t also has
labor contracts that g#arantee employees in each mine a f#ll dayBs pay for each day or fraction of a day the mine is open.
Cetermine the n#mber of days each mine sho#ld be operated d#ring the #pcoming wee) if :niversal ;ines is to f#lfill its
commitment at minim#m total cost.
"he first constraint in the above problem is
4x1 + 6x2 + x3 54
"he left&hand side of this ine'#ality represents the combined o#tp#t of high&grade ore from three mines, while the right&hand side
is the minim#m tonnage of s#ch ore re'#ired to meet contract#al obligations. "his ine'#ality is transformed into the e'#ation
4x1 + 6x2 + x3 x4 = 54
by s#btracting the s#rpl#s variable x4 from the left&hand side of the ine'#ality. 6ere x4 represents the amo#nt of high&grade ore
mined over and above that needed to f#lfill the contract.
1&-4
Generating an Initial Feasible Solution
After all linear constraints with nonnegative right&hand sides! have been transformed into e'#alities by introd#cing slac) and
s#rpl#s variables where necessary, add a new variable, called an artificial variable, to the left&hand side of each constraint
e'#ation that does not contain a slac) variable. $ach constraint e'#ation will then contain either one slac) variable or one artificial
variable. A nonnegative initial sol#tion to this new set of constraints is obtained by setting each slac) variable and each artificial
variable e'#al to the right&hand side of the e'#ation in which it appears and setting all other variable, incl#ding the s#rpl#s
variable, e'#al to 7ero.
Example: "he set of constraints
x1 + 2x2 3
4x1 + 5x2 6
7x1 + 8x2 = 15
is transformed into a system of e'#ations by adding a slac) variable, x3, to the left&hand side of the first constraint and s#btracting
a s#rpl#s variable, x4, from the left&hand side of the second constraint. "he new system is
x1 + 2x2 + x3 = 3
4x1 + 5x2 x4 = 6
7x1 + 8x2 = 15
*f now artificial variables x5 and x6 are respectively added to the left&hand sides of the last two constraints in system, the
constraints witho#t a slac) variable, the res#lt is
x1 + 2x2 + x3 = 3
4x1 + 5x2 x4 + x5 = 6
7x1 + 8x2 + x6 = 15
A nonnegative sol#tion to this last system is x3 = 3, x5 = 6, x6 = 15, and x1 = x2 = x4 = 0. Dotice, however, that x1 = 0, x2 = 0 is
not a sol#tion to the original set of constraints.!
Penalty Costs
"he introd#ction of slac) and s#rpl#s variables alters neither the nat#re of the constraints nor the obEective. Accordingly, s#ch
variables are incorporated into the obEective f#nction with 7ero coefficients. Artificial variables, however, do change the nat#re of
the constraints. (ince they are added to only one side of an e'#ality, the new system is e'#ivalent to the old system of constraints
if and only if the artificial variables are 7ero. "o g#arantee s#ch assignments in the optimal sol#tion in contrast to the initial
sol#tion!, artificial variables are incorporated into the obEective f#nction with very large positive coefficients in a minimi7ation
program or very large negative coefficients in a ma%imi7ation program. "hese coefficients, denoted by either or , where
is #nderstood to be a large positive n#mber, represent the severe! penalty inc#rred in ma)ing a #nit assignment to the artificial
variables.
Standard atrix Form:
;inimi7e: ! = x1 + 2x2 + 3x3
(#bEect to: 3x1 + 4x3 5
5x1 + x2 + 6x3 = 7
8x1 + "x3 2
<ith: all variables nonnegative
Adding a slac) variable x4 to the left&hand side of the first constraint, s#btracting a s#rpl#s variable x5 from the left&hand side of
the third constraint, and then adding an artificial variable x6 only to the left&hand side of the third constraint, we obtain the
program
;inimi7e: ! = x1 + 2x2 + 3x3 + 0x4 + 0x5 + x6
(#bEect to: 3x1 + 4x3 + x4 = 5
5x1 + x2 + 6x3 = 7
8x1 + "x3 # x5 + x6 = 2
<ith: all variables nonnegative
"his program is in standard form, with an initial feasible sol#tions x4 = 5, x2 = 7, x6 = 2, x1 = x3 = x5 = 0.
define
$ F Gx1, x2, x3, x4, x5, x6H
%
& F G1, 2, 3, 0, 0, H
%
/&-4
1
1
1
]
1

1 1 4 3 4 A
4 4 4 2 1 -
4 4 1 , 4 .
'
1
1
1
]
1

/
1
-
(
1
1
1
]
1

2
/
,
4
x
x
x
$
*n this case, x2 can be #sed to generate the initial sol#tion rather than adding an artificial variable to the second constraint to
achieve the same res#lt. *n general, whenever a variable appears in one and only one constraint e'#ation, and there with a positive
coefficient, that variable can be #sed to generate part of the initial sol#tion by first dividing the constraint e'#ation by the positive
coefficient and then setting the variable e'#al to the right&hand side of the e'#ation= an artificial variable need not be added to the
e'#ation.
!"e Simplex !ableau
"he simple% method is a matri% proced#re for solving linear programs in the standard form
?ptimi7e: ! = &
%
$
(#bEect to: '$ = (
<ith: $ 0
<here ( 0 and a basic feasible sol#tion $0 is )nown. (tarting with $0, the method locates s#ccessively other basic feasible
sol#tions having better val#es of the obEective, #ntil the optimal sol#tion is obtained. Ior minimi7ation programs, the simple%
method #tili7es tablea#, in which &0 designates the cost vector associated with the variables in $0.
( & ' & &
( ' & $
&
$
% % %
%
%
4 4
4 4

Ior ma%imi7ation programs, tablea# applies if the elements of the bottom row have their signs reversed.
Example: Ior the minimi7ation program of above problem, &0 = G4, /, ;H
%
. "hen
[ ] [ ]
1
1
1
]
1


1 1 4 3 4 A
4 4 4 2 1 -
4 4 1 , 4 .
, / , 4 , 4 , 4 , . , / , 1
4
' & &
% %
[ ] [ ] + + , 4 , 3 1/ , / , A 14 , 4 , 4 , . , / , 1
[ ] 4 , , 4 , 3 3 , 4 , A 3
[ ] ( &
%
/ 1,
/
1
-
, / , 4
4

1
1
1
]
1


And tablea# becomes

x
x
x

x x x x x x
/ 1, 4 4 3 3 4 A 3
/ 1 1 4 3 4 A
1 4 4 4 2 1 - /
- 4 4 1 , 4 . 4
4 4 . / 1
2
/
,
2 - , . / 1

# !ableau Simpli$ication
Ior each j j = 1, /, J , n!, define !j =
%
&
4
'j, the dot prod#ct of &0 with the j th col#mn of '. "he j th entry in the last row of
"ablea# is cj !j or, for a ma%imi7ation program, !j cj!, where cj is the cost in the second row of the tablea#, immediately above
'j. ?nce this last row has been obtained, the second row and second col#mn of the tablea#, corresponding to &
%
and &0,
respectively, become s#perfl#o#s and may be eliminated.
!"e Simplex et"od
.&-4
(tep 1: Locate the most negative n#mber in the bottom row of the simple% tablea#, e%cl#ding the last col#mn, and call the col#mn
in which this n#mber appears the wor) col#mn. *f more than one candidate for most negative n#mbers e%ists, choose one.
(tep /: Iorm ratios by dividing each positive n#mber in the wor) col#mn, e%cl#ding the last row, into the element in the same row
and last col#mn. Cesignate the element in the wor) col#mn that yields the smallest ration as the pivot element. *f more
that one element yields the same smallest ration, choose one. *f no element in the wor) col#mn is positive, the program
has no sol#tion.
(tep .: :se elementary row operations to convert the pivot element to 1 and then to red#ce all other elements in the wor) col#mn
to 4.
(tep ,: Keplace the x&variable in the pivot row and first col#mn by the x&variable in the first row and pivot col#mn. "his new first
col#mn is the c#rrent set of basic variables.
(tep -: Kepeat (teps 1 thro#gh , #ntil there are no negative n#mbers in the last row, e%cl#ding the last col#mn.
(tep 2: "he optimal sol#tion is obtained by assigning to each variable in the first col#mn that val#e in the corresponding row and
last col#mn. All other variables are assigned the val#e 7ero. "he associated 7L, the optimal val#e of the obEective
f#nction, is the n#mber in the last row and last col#mn for a ma%imi7ation program, b#t the negative of this n#mber for a
minimi7ation program.
odi$ication $or Programs %it" #rti$icial Variables
<henever artificial variables are part of the initial sol#tion $4, the last row of "ablea# will contain the penalty cost . "o
minimi7e ro#ndoff error, the following modifications are incorporated into the simple% method= the res#lting algorithm is the two&
phase method.
@hange 1: "he last row of "ablea# is decomposed into two rows, the first of which involves those terms not containing , while
the second involves the coefficients of ; in the remaining terms.
Example: "he last row of the tablea# in the last e%ample is
/ 1, 4 , , 4 , 3 3 , 4 , A 3
:nder @hange 1 it wo#ld be transformed into the two rows
1, 4 4 4 3 4 3
/ 4 1 4 3 , 4 A
@hange /: (tep 1 of the simple% method is applied to the last row created in @hange 1 followed by (tep /, ., and ,!, #ntil this
row contains no negative elements. "he (tep 1 is applied to those elements in the ne%t&to&last row that are positioned
over 7eros in the last row.
@hange .: <henever an artificial variable ceases to be basic M i.e., is removed from the first col#mn of the tablea# as a res#lt of
(tep , M it is deleted from the top row of the tablea#, as is the entire col#mn #nder it. "his modification simplifies hand
calc#lations b#t is not implemented in many comp#ter programs.!
@hange ,: "he last row can be deleted from the tablea# whenever it contains all 7eros.
@hange -: *f non7ero artificial variables are present in the final basic set, then the program has no sol#tion. *n contrast, 7ero&
val#ed artificial variables may appear as basic variables in the final sol#tion when one or more of the original constraint
e'#ations is red#ndant.!
Solved Problems
Problem &
;a%imi7e: ! = x1 + "x2 + x3
(#bEect to: x1 + 2x2 + 3x3 "
3x1 + 2x2 +2x3 15
<ith: all variables nonnegative
"his program is p#t into matri% standard form by first introd#cing slac) variables x4 and x5 in the first and second constraint
ine'#alities, and then defining
$ F Gx1, x2, x3, x4, x5H
%
& F G1, ",1, 0, 0H
%
1
]
1

1 4 / / .
4 1 . / 1
'
1
]
1

1-
3
(
1
]
1

-
,
4
x
x
$
"he costs associated with the components of $0, the slac) variable, are 7ero= hence &0 F G4, 4H
"
.
"ablea# becomes
,&-4
1- 1 4 / / . 4
3 4 1 . / 1 4
4 4 1 3 1
-
,
- , . / 1
x
x
x x x x x
"o comp#te the last row of this tablea#, we #se the tablea# simplification and first calc#late each !j by inspection: it is the dot
prod#ct of col#mn / and the j th col#mn of '. <e then s#btract the corresponding cost cj from it ma%imi7ation program!. *n this
case, the second col#mn is 7ero, and so !j cj = 0 cj = cj. 6ence, the bottom row of the tablea#, e%cl#ding the last element, is
E#st the negative of row /. "he last element in the bottom row is simply the dot prod#ct of col#mn / and the final, (&col#mn, and
so it too is 7ero. At this point, the second row and second col#mn of the tablea# are s#perfl#o#s. $liminating them, we obtain
tablea# as the complete initial tablea#.
4 4 4 1 3 1 !
1- 1 4 / / .
3 4 1 . / 1
4 4 1 3 1
-
L
,
- , . / 1

j j
c !
x
x
x x x x x
<e are now ready to apply the simple% method. "he most negative element in the last row of tablea# is 3, corresponding to the
x2&col#mn= hence this col#mn becomes the wor) col#mn. Iorming the ratios 35/ = ,.- and 1-5/ = 1.-, we find that the element /,
mar)ed b the asteris) in the tablea#, is the pivot element, since it yields the smallest ratio. "hen, applying (teps . and , to the
tablea#, we obtain ne%t tablea#. (ince the last row of the last table contains no negative elements, it follows from (tep 2 that the
optimal sol#tion is x2 = ")2, x5 = 6, x1 = x3 = x4 = 0, with ! = 81)2.
Problem '
;inimi7e: ! = 80x1 + 60x2
(#bEect to: 0*20x1 + 0*32x2 0*25
x1 + x2 = 1
with: x1 and x2 nonnegative
Adding a slac) variable x3 and an artificial variable x4 to the first and second constraints, respectively, we convert the program to
standard matri% form, with
$ F Gx1, x2, x3, x4H
%
& F G80,60,0, H
%
1
]
1

1 4 1 1
4 1 ./ . 4 /4 . 4
'
1
]
1

1
/- . 4
(
1
]
1

,
.
4
x
x
$
(#bstit#ting these matrices, along with &0 F G4, ;H
"
, we obtain initial tablea#.

x
x

x x x x
4 4 24 A4
1 1 4 1 1
/- . 4 4 1 ./ . 4 /4 . 4 4
4 24 A4
-
.
, . / 1
(ince the bottom row involves ;, we apply @hange 1= the res#lting tablea# is the initial tablea# for the two&phase method.
1 4 4 1 1
4 4 4 24 A4 !
1 1 4 1 1
/- . 4 4 1 ./ . 4 /4 . 4
L
,
.
, . / 1

j j
! c
x
x
x x x x
:sing both (tep 1 of the (imple% method and @hange /, we find that the most negative element in the last row of tablea#
e%cl#ding the last col#mn! is M 1, which appears twice. Arbitrarily selecting the x1&col#mn as the wor) col#mn, we form the
-&-4
ratios 4./-54./4 = 1./- and 151 = 1. (ince the element 1, starred in tablea#, yields the smallest ratio, it becomes the pivot. "hen,
applying (tep . and , and @hange . to tablea# we generate ne%t tablea#.
4 4 4 4
A4 4 /4 4
1 4 1 1
4- . 4 1 1/ . 4 4
1
L
.
. / 1

x
x
x x x
?bserve that x1 replaces the artificial variable x4 in the first col#mn of tablea#, so that the entire x4&col#mn is absent from the
tablea#. Dow, with no artificial variables in the first col#mn and with @hange . implemented, the last row of the tablea# sho#ld be
all 7eros. *t is= and by @hange , this row may be deleted, giving
A4 4 /4 4
as the new last row of tablea#.
Kepeating (teps 1 thro#gh ,, we find that the x2&col#mn is the new wor) col#mn, the starred element in the tablea# is the new
pivot, and the elementary row operations yield the ne%t tablea#, in which all calc#lations have been ro#nded to fo#r significant
fig#res. (ince the last row of the tablea#, e%cl#ding the last col#mn, contains no negative elements, it follows from (tep 2 that x1
= 0*5833, x2 = 0*4167, x3 = x4 = 0, +it, ! = 71*67
21 . 11 1 . 122 4 4
-A.. . 4 ... . A 4 1
,121 . 4 ... . A 1 4
1
/
. / 1

x
x
x x x
Problems:
:se the simple% or two&phase method to solve the following problems.
..1-
;a%imi7e: ! = x1 + x2
s.t. x1 + 5x2 5
2x1 + x2 4
<ith: x1, x2 nonnegative
..12
;a%imi7e: ! = 3x1 + 4x2
s.t. 2x1 + x2 6
2x1 + 3x2 "
<ith: x1, x2 nonnegative
..11
;inimi7e: ! = x1 + 2x2
s.t. x1 + 3x2 11
2x1 + x2 "
<ith: x1, x2 nonnegative
..1A
;a%imi7e: ! = Mx1 M x2
s.t. x1 + 3x2 5000
2x1 + x2 12000
<ith: x1, x2 nonnegative
..13
;a%imi7e: ! = 2x1 + 3x2 + 4x3
s.t. x1 + x2 + x3 1
x1 + x2 + 2x3 = 2
3x1 + 2x2 + x3 4
<ith: x1, x2, x3 nonnegative
!"e (ual Simplex et"od
"he reg#lar! simple% method moves the initial feasible b#t nonoptimal sol#tion to an optimal sol#tion while maintaining
feasibility thro#gh an iterative proced#re. ?n the other hand, the d#al simple% method moves the initial optimal b#t infeasible
sol#tion to a feasible sol#tion while maintaining optimality thro#gh an iterative proced#re.
Iterative procedure o$ t"e (ual Simplex et"od:
(tep 1: Kewrite the linear programming problem by e%pressing all the constraints in form and transforming them into e'#ations
thro#gh slac) variables.
(tep /: $%hibit the above problem in the form of a simple% tablea#. *f the optimality condition is satisfied and one or more basic
variables have negative val#es, the d#al simple% method is applicable.
(tep .: Ieasibility @ondition: "he basic variable with the most negative val#e becomes the departing variable C.+.!. @all the row
in which this val#e appears the wor) row. *f more than one candidate for C.+. e%ists, choose one.
(tep ,: ?ptimality @ondition: Iorm ratios by dividing all b#t the last element of the last row of cj !j val#es minimi7ation
problem! or the !j cj val#es ma%imi7ation problem! by the corresponding negative coefficients of the wor) row. "he
nonbasic variable with the smallest absol#te ration becomes the entering variable $.+.!. Cesignate this element in the
wor) row as the pivot element and the corresponding col#mn the wor) col#mn. *f more than one candidate for $.+.
e%ists, choose one. *f no element in the wor) row is negative, the problem has no feasible sol#tion.
2&-4
(tep -: :se elementary row operations to convert the pivot element to 1 and then to red#ce all the other elements in the wor)
col#mn to 7ero.
(tep 2: Kepeat steps . thro#gh - #ntil there are no negative val#es for the basic variables.
1&-4
Solved Problems
Problem &
:se the d#al simple% method to solve the following problem.
;inimi7e: ! = 2x1 + x2 + 3x3
s.t. x1 2x2 + x3 4
2x1 + x2 + x3 8
x1 x3 0
<ith: all variables nonnegative
$%pressing all the constraints in the form and adding the slac) variables, the problem becomes:
;inimi7e: ! = 2x1 + x2 + 3x3 + 0x4 + 0x5 + 0x6
s.t. x1 + 2x2 x3 + x4 = 4
2x1 + x2 + x3 + x5 = 8
x1 + x3 + x5 = 0
<ith: all variables nonnegative
4 4 4 4 . 1 / !
4 1 4 4 1 4 1
A 4 1 4 1 1 /
, 4 4 1 1 / 1
2
-
L
,
2 - , . / 1
j j
! c
x
x
x
x x x x x x


(ince all the cj M !j! val#es are nonnegative, the above sol#tion is optimal. 6owever, it is infeasible beca#se it has a nonpositive
val#e for the basic variable x4. (ince x4 is the only nonpositive variable, it becomes the departing variable C.+.!.

. / : ratios absol#te
4 4 1 1 / 1 : row
4 4 4 . 1 / : !
,
2 - , . / 1
x
r-+ ! c
x x x x x x
j j
(ince x1 has the smallest absol#te ratio, it becomes the entering variable $.+.!. "h#s the element M 1, mar)ed by the asteris),
becomes the pivot element. :sing elementary row operations, we obtain the ne%t tablea#.
A 4 4 / 1 - 4 : !
, 1 4 1 / / 4
4 4 1 / 1 - 4
, 4 4 1 1 / 1
2
-
1
2 - , . / 1


j j
! c
x
x
x
x x x x x x
(ince all the variables have nonnegtive val#es, the above optimal sol#tion is feasible. "he optimal and feasible sol#tion is x1 = 4,
x2 = 0, x3 = 0, with ! = 8.
Problem '
:se the d#al simple% method to solve the following problem.
;a%imi7e: ! = 2x1 3x2
s.t. x1 + x2 2
2x1 + x2 10
x1 + x2 8
with: x1 and x2 nonnegative
$%pressing all the constraints in the form and adding the slac) variables, the problem becomes:
;a%imi7e: ! = 2x1 3x2
s.t. x1 + x2 2
2x1 + x2 10
x1 + x2 8
with: all variables nonnegative
A&-4
4 4 4 4 . / : !
A 1 4 4 1 1
14 4 1 4 1 /
/ 4 4 1 1 1
-
,
L
.
- , . / 1
j j
c !
x
x
x
x x x x x


(ince all the cj M !j! val#es are nonnegative, the above sol#tion is optimal. 6owever, it is infeasible beca#se it has a nonpositive
val#e for the basic variable x3. (ince x3 is the only nonpositive variable, it becomes the departing variable C.+.!.

. / : ratios absol#te
4 4 1 1 1 : row
4 4 4 . / : row !
.
- , . / 1
x
c !
x x x x x
j j
(ince x1 has the smallest absol#te ration, it becomes the entering variable $.+.!. "h#s the element M 1, mar)ed by the asteris), is
the pivot element. :sing elementary row operations, we obtain the ne%t tablea#.
, 4 4 / 1 4 : !
2 1 4 1 4 4
2 4 1 / 1 4
/ 4 4 1 1 1
-
,
1
- , . / 1

j j
c !
x
x
x
x x x x x
(ince all the variables have nonnegative val#es, the above optimal sol#tion is feasible. "he optimal and feasible sol#tion is x1 = /,
x2 = 4, with 7 = ,.
Problems:
...1
;inimi7e: ! = 3x1 9 x2
s.t. 3x1 9 x2 4
x1 M x2 0
with: x1, x2 nonnegative
.../
;inimi7e: ! = 5x1 + 3x2 + x3
s.t. M x1 9 x2 9 x3 2
2x1 9 x2 M x3 4
with: all variables nonnegative
....
;inimi7e: ! = x1 + 2x2 + 3x3
s.t. x1 2x2 + x3 1
3x1 4x2 + 6x3 8
2x1 4x2 + 2x3 2
with: all variables nonnegative
...,
;a%imi7e: ! = 4x1 3x2
s.t. x1 9 x2 2
x1 + 2x2 8
with: x1, x2 nonnegative
...-
;a%imi7e: ! = 3x1 4x2 5x3
s.t. x1 + x2 10
x1 + 3x2 + x3 "
x2 + x3 4
with: all variables nonnegative
...2
;inimi7e: ! = 3x1 + "x2
s.t. x1 + 3x2 6
2x1 + 3x2 "
with: x1, x2 nonnegative
3&-4
)inear Programming: (uality and Sensitivity #nalysis
$very linear program in the variables x1, x2, J , xn has associated with it another linear program in the variables +1, +2, J , +.
where m is the n#mber of constraints in the original program!, )nown as its d#al. "he original program, called the primal,
completely determines the form of its d#al.
Symmetric (uals
"he d#al of a primal! linear program in the nonstandard! matri% form
minimi7e: ! = &
%
$
1! s.t. '$ (
with: $ 0
is the linear program
ma%imi7e: ! = (
%
/
/! s.t. '
%
/ &
with: / 0
@onversely, the d#al of the second program is the first program.
Nrograms 1! and /! are symmetrical in that both involve nonnegative variables and ine'#ality constraints= the +1, +2, J , +. are
)nown as the s0..etric 12als of each other. "he d#al variables are sometimes called s,a1-+ c-sts.
(ual solutions
!"eorem & *(uality !"eorem+: *f an optimal sol#tion e%ists to either the primal or symmetric d#al program, then the other
program also has an optimal sol#tion and the two obEective f#nctions have the same optimal val#e.
*n s#ch sit#ation, the optimal sol#tion to the primal d#al! is fo#nd in the last row of the final simple% tablea# for the d#al
primal!, in those col#mns associated with the slac) or s#rpl#s variables. (ince the sol#tions to both programs are obtained by
solving either one, it may be comp#tationally advantageo#s to solve a programBs d#al rather than the program itself.
!"eorem ' *Complementary Slackness Principle+: >iven that the pair of symmetric d#als have optimal sol#tions, then if the k&
th constraint of one system holds as an ine'#ality M i.e., the associated slac) or s#rpl#s variable is positive M the k&th
component of the optimal sol#tion of tits symmetric d#al is 7ero.
,nsymmetric (auls
Ior primal programs in standard matri% form, d#als may be defined as follows:
Primal (ual
;inimi7e: ! = &
%
$
s.t. '$ = (
with: O 0
.!
;a%imi7e: ! = (
%
/
s.t. '
%
/ &
,!
;a%imi7e: ! = &
%
$
s.t '$ = (
with: $ 0
-!
;inimi7e: ! = (
%
/
s.t. '
%
/ & 2!
@onversely, the d#als of programs ,! and 2! are defined as programs .! and -!, respectively. (ince the d#al of a program in
standard form is not itself in standard form, these d#als are 2ns0..etric. "heir forms are consistent with and a direct conse'#ence
of the definition of symmetric d#als.
"heorem 1 is valid for #nsymmetric d#als too. 6owever, the sol#tion to an #nsymmetric d#al is not, in general, immediately
apparent from the sol#tion to the primal= the relationships are
1
0
%
0
%
' & /

L
or
0
1 %
0
& ' /

! L
1!
1 %
0
%
0
%
' ( $

! L
or
0
1
0
( ' $

L
A!
*n 1!, &0 and '0 are made #p of those elements of & and ', in either program .! or -!, that correspond to the basic variables in
$L= in A!, (0 and '0 are made #p of those elements of ( and ', in either program ,! or 2!, that correspond to the basic variables
in /L.
Solved Problems
1. Cetermine the symmetric d#al of the program
;inimi7e: ! = 5x1 + 2x2 + x3
s.t. 2x1 + 3x2 + x3 20
6x1 + 8x2 + 5x3 30
7x1 + x2 + 3x3 40
x1 + 2x2 + 4x3 50
with: all variables nonnegative
14&-4
ma%imi7e: ! = 20+1 + 30+2 + 40+3 + 50+4
2+1 + 6+2 + 7+3 + +4 5
3+1 + 8+2 + +3 + 2+4 2
+1 + 5+2 + 3+3 + 4+4 1
with: all variables nonnegative
/. Cetermine the symmetric d#al of the program
;a%imi7e: ! = 2x1 + x2
s.t. x1 + 5x2 10
x1 + 3x2 6
2x1 + 2x2 8
with: all variables nonnegative
minimi7e: ! = 10+1 + 6+2 + 8+3
s.t. +1 + +2 + 2+3 2
5+1 + 3+2 + 2+3 1
with: all variables nonnegative
sol#tion for primal and d#al problems:

variables slack

4 4 4 4 1 / : !
A 1 4 4 / /
2 4 1 4 . 1
14 4 4 1 - 1
L
-
,
.
- , . / 1

j j
c !
x
x
x
x x x x x
A 1 4 4 1 4
, / 1 4 4 1 1
/ / 1 1 4 / 4
2 / 1 4 1 , 4
1
,
.
- , . / 1
x
x
x
x x x x x


12al t,e t- s-l2ti-n


variables s2r3l2s

. 4 4 1 1 , , 2
4 4 4 4 4 A 2 14 !
1 1 4 1 4 / . -
/ 4 1 4 1 / 1 1
1
2
1 2 - , . / 1

j j
! c
+
+
+ + + + + + +
A 4 , 4 / 2
1 4 / 1 1 / 1 / 1
1 1 1 4 - ,
.
-
- , . / 1


+
+
+ + + + +

3ri.al t,e t- s-l2ti-n

/. Cetermine the d#al of the program
;a%imi7e: ! = x1 + 3x2 2x3
s.t. 4x1 + 8x2 + 6x3 = 25
7x1 + 5x2 + "x3 = 30
with: all variables nonnegative
"he primal program can be solved by the two&phase method if artificial variables %, and %-, respectively, are first added to the
left&hand sides of the constraints e'#ations. (imple% tablea#% res#lt.
55 0 0 15 13 11
0 0 0 2 3 1 c !
30 1 0 " 5 7 x
25 0 1 6 8 4 x
x x x x x
j j
5
4
5 4 3 2 1

!
777 7 668 3 0 0
1"3 3 167 1 0 1 x
528 1 1668 0 1 0 x
x x x
1
2
3 2 1
. .
. .
. .
"he primal program has the form -!= its #nsymmetrid d#al is given by 2! as
;inimi7e: ! = 25+1 + 30+2
s.t. 4+1 + 4+2 1
8+1 + 5+2 2
6+1 + "+2 2
11&-4
4 0 0 0 1 1 12 12 12 12
0 0 0 0 0 0 30 30 25 25 ! c
2 0 0 1 0 0 " " 6 6 +
3 1 0 0 1 0 5 5 8 8 +
1 0 1 0 0 1 7 7 4 4 +
+ + + + + + + + +
j j
"
11
10
11 10 " 8 7 6 5 4 3





!
778 7 0 528 1 1"5 3 0 0 0 0
667 3 1 1667 0 167 1 0 0 0 0 +
1111 0 0 1111 0 2222 0 1 1 0 0 +
4444 0 0 1"44 0 138" 0 0 0 1 1 +
+ + + + + + +
"
6
3
" 8 7 6 5 4 3
. . .
. . .
. . .
. . .

+1 = +3 +4 = 0*4444
+2 = +5 +6 = 0*1111
"o verify, we note that the basic variables in $
L
are x1 and x2
[ ] [ ] [ ] [ ] 1111 0 4444 0 36 4 36 16
36 4 36 7
36 8 36 5
3 1
5 7
8 4
3 1 /
1
%
. , . , , ,
L

1
]
1

1
]
1

"o verify, we note that the basic variables in /


L
are +3, +6, and +"
[ ] [ ] [ ] [ ] 0 528 1 1"4 3 0 36 55 36 115
1 36 6 36 42
0 36 4 36 8
0 36 7 36 5
0 30 25
1 " 6
0 5 8
0 7 4
0 30 25 $
1
%
, . , . , , , , , ,
L

1
1
1
]
1


1
1
1
]
1

Sensitivity #nalysis
"he scope of linear programming does not end at finding the optimal sol#tion to the linear model of a real&life problem.
(ensitivity analysis of linear programming contin#es with the optimal sol#tion to provide additional practical insight of the model.
(ince this analysis e%amines how sensitive the optimal sol#tion is to changes in the coefficients of the LN model, it is called
sensitivity analysis. "his process is also )nown as postoptimality analysis beca#se it starts after the optimal sol#tion is fo#nd.
(ince we live in a dynamic world where changes occ#r constantly, this st#dy of the effects on the sol#tion d#e to changes in the
data of a problem is very #sef#l
*n general, we are interested in finding the effects of the following changes on the optimal LN sol#tion:
i! @hanges in profit5#nit or cost5#nit coefficients! of the obEective f#nction.
ii! @hanges in the availability of reso#rces or capacities of prod#ction5service center or limits on demands
re'#irements vector or K6( of constraints!.
iii! @hanges in reso#rce re'#irements5#nits of prod#cts or activities technological coefficients of variables! in
constraints.
iv! Addition of a new prod#ct or activity variable!.
v! Addition of a new constraint.
"he sensitivity analysis will be disc#ssed for linear programs of the form:
;a%imi7e: ! = &
%
$
s.t. '$ (
with $ 0
where $ is the col#mn vector of #n)nowns= &
%
is the row vector of the corresponding costs cost vector!= ' is the coefficient
matri% of the constraints matri% of technological coefficients!= and ( is the col#mn vector of the right&hand sides of the
constraints re'#irements vector!.
Example &:
;a%imi7e: ! = 20x1 9 10x2
s.t. x1 9 2x2 40
3x1 9 2x2 60
with: x1 and x2 nonnegative
"his program is p#t into the following standard form by introd#cing the slac) variables %. and %,:
;a%imi7e: ! = 20x1 9 10x2 9 0x3 9 0x4
x1 9 2x2 9 x3 = 40
3x1 9 2x2 9 x4 = 60
with: all variables nonnegative
"he sol#tion for this problem is s#mmari7ed as follows:
*nitial (imple% "ablea#:
1/&-4
0 0 0 10 20 c !
60 1 0 2 3 0 x
40 0 1 2 1 0 x
0 0 10 20
x x x x
j j
4
3
4 3 2 1
!
Iinal (imple% "ablea#:
400 3 20 0 3 10 0 c !
20 3 1 0 3 2 1 20 x
20 3 1 1 3 4 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1
!

(ince the last row of the above tablea# contains no negative elements, the optimal sol#tion is x1 = 20, x2 = 0, with ! = 400
Example &-& ;odification of the cost &
%
a! @oefficients of the nonbasic variables.
Let the new val#e of the cost coefficient corresponding to the nonbasic variable x2 be 15 instead of 10.
"he corresponding simple% tablea# is
400 3 20 0 3 5 0 c !
20 3 1 0 3 2 1 20 x
20 3 1 1 3 4 0 0 x
0 0 15 20
x x x x
j j
1
3
4 3 2 1

!
(ince !2 M c2! P 0, the new sol#tion is not optimal. "he reg#lar simple% method is #sed to reoptimi7e the problem, starting with x2
as the entering variable.
"he new optimal tablea# is
425 4 25 4 5 0 0 c !
10 2 1 2 1 0 1 20 x
15 4 1 4 3 1 0 15 x
0 0 15 20
x x x x
j j
1
2
4 3 2 1
!

"he optimal sol#tion is x1 = 10, x2 = 15, with ! = 425.


b! @oefficients of the basic variables.
Let the cost coefficient of the basic variable x1 be changed from 20 to 10. "hen the simple% tablea# becomes
400 3 10 0 3 10 0 c !
20 3 1 0 3 2 1 10 x
20 3 1 1 3 4 0 0 x
0 0 10 10
x x x x
j j
1
3
4 3 2 1

!
(ince !2 M c2! P 0, the new sol#tion is not optimal. "he reg#lar simple% method is resorted to for reoptimi7ation, first by entering
x2. "he new optimal tablea# is
250 2 5 2 5 0 0 c !
10 2 1 2 1 0 1 20 x
15 4 1 4 3 1 0 10 x
0 0 10 10
x x x x
j j
1
2
4 3 2 1
!

"he optimal sol#tion is x1 = 10, x2 = 15, with ! = 250.


Example &-' ;odification of the re'#irements vector (
Let the K6( of the second constraint be changed from 24 to 1.4
"hen $s = 4
51
( becomes

,
_

,
_

,
_

,
_

3
1
3
1
1
3
4 3
3
1 3 0
4 0
3 1 0
3 1 1
x
x
1.&-4
(ince x3 P 0, the new sol#tion is not feasible. "he d#al simple% method #sed to clear the infeasibility starting with the following
tablea#:
67 866 67 6 0 33 3 0 c !
33 43 33 0 0 67 0 1 20 x
33 3 33 0 1 33 1 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1
. . . !
. . .
. . .


"he new final tablea# is
800 0 20 30 0 c !
40 0 1 2 1 20 x
10 1 3 4 0 0 x
0 0 10 20
x x x x
j j
1
4
4 3 2 1
!

"he optimal and feasible sol#tion is x1 = 40, x2 = 0, with ! = 800.
Example &-. ;odification of the matri% of coefficients '
"he problem becomes more complicated, when the technological coefficients of the basic variables are considered. "his is
beca#se here the matri% #nder the starting sol#tion changes. *n this case, it may be easier to solve the new problem than resort to
the sensitivity analysis approach. "herefore o#r analysis is limited to the case of the coefficients of nonbasic variables only.
Let the technological coefficients of x2 be changed from 2, 2!
%
to 2, 1!
%
.
"hen the new technological coefficients of x2 in the optimal simple% tablea# of the original primal problem are given by

,
_

,
_

,
_

3
1
3
5
1
2
3
1
0
3
1
1
6 4
2
1
6ence the new simple% tablea# becomes
400 3 20 0 3 10 0 c !
20 3 1 0 3 1 1 20 x
20 3 1 1 3 5 0 0 x
0 0 10 20
x x x x
j j
1
3
4 3 2 1

!
(ince here !2 M c2! P 0, the new sol#tion is not optimal. Again the reg#lar simple% method is resorted to for reoptimi7ation, first
by entering x2. "he new optimal tablea# is
440 6 2 0 0 c !
16 5 2 5 1 0 1 20 x
12 5 1 5 3 1 0 10 x
0 0 10 20
x x x x
j j
1
2
4 3 2 1
!

"he optimal sol#tion is x1 = 16, x2 = 12, with ! = 440.


Example &-/ Addition of a variable
Let a new variable xk be added to the original problem. "his is accompanied by the addition to ' of a col#mn 6k = 3, 1!
%
and to &
%
of a component ck = 30. "h#s the new problem becomes
;a%imi7e: ! = 20x1 9 10x2 + 30xk
s.t. x1 9 2x2 9 3xk 40
3x1 9 2x2 9 xk 60
with: all variables nonnegative
"hen the technological coefficients of xk in the optimal tablea# are

,
_

,
_

,
_

3
1
3
8
1
3
3
1
0
3
1
1
6 4
k
1
"he corresponding !k M ck! = 852!0! 9 153!20! M 30 = 7053
"h#s the modified simple% tablea# is
1,&-4
400 3 20 0 3 70 3 10 0 c !
20 3 1 0 3 1 3 2 1 20 x
20 3 1 1 3 8 3 4 0 0 x
0 0 30 10 20
x x x x x
j j
1
3
4 3 k 2 1

!
Dow entering the variable xk, the reg#lar simple% method is applied to obtain the following optimal tablea#.
575 8 30 8 70 0 15 0 c !
2 35 8 3 8 1 0 2 1 1 20 x
2 15 8 1 8 3 1 2 1 0 30 x
0 0 30 10 20
x x x x x
j j
1
k
4 3 k 2 1
!

"he optimal sol#tion is x1 = 17*5, x2 = 0, xk = 7*5, with ! = 575.


Example &-0 Addition of a constraint
*f a new constraint added to the system is not active, it is called a secondary or red#ndant constraint, and the optimality of the
problem remains #nchanged. ?n the other hand, if the new constraint is active, the c#rrent optimal sol#tion becomes infeasible.
Let #s consider the case of the addition of an active constraint, 2x1 9 3x2 50 to the original problem. "he c#rrent optimal sol#tion
x1 = 20, x2 = 0! does not satisfy the above new constraint and hence becomes infeasible. "herefore, add the new constraint to the
c#rrent optimal tablea#. "he new slac) variable is x5. "he new simple% tablea# is
400 0 3 20 0 3 10 0 c !
50 1 0 0 3 2 0 x
20 0 3 1 0 3 2 1 20 x
20 0 3 1 1 3 4 0 0 x
0 0 0 10 20
x x x x x
j j
5
1
3
5 4 3 2 1
! +

Qy #sing the row operations, the coefficient of x1 in the new constraint is made 7ero. "he modified tablea# becomes
400 0 3 20 0 3 10 0 c !
10 1 3 2 0 3 5 0 0 x
20 0 3 1 0 3 2 1 20 x
20 0 3 1 1 3 4 0 0 x
0 0 0 10 20
x x x x x
j j
5
1
3
5 4 3 2 1
! +

"he d#al simple% method is #sed to overcome the infeasibility by departing the variable x5. "he new tablea# is
380 2 8 0 0 0 c !
6 5 3 5 2 0 1 0 10 x
16 5 2 5 3 0 0 1 20 x
12 5 4 5 1 1 0 0 0 x
0 0 0 10 20
x x x x x
j j
2
1
3
5 4 3 2 1
! +

"he above tablea# gives the optimal and feasible sol#tion as x1 = 16, x2 = 6, x3 = 12, with ! = 380.
!"e 1evised Simplex et"od
@onsider the following linear programming problem in standard matri% form:
;a%imi7e: ! = &
%
$
s.t. '$ = (
with: $ 0
where $ is the col#mn vector of #n)nowns, incl#ding all slac), s#rpl#s, and artificial variables= &
%
is the row vector of
corresponding costs= ' is the coefficient matri% of the constraint e'#ations= and ( is the col#mn vector of the right&hand side of the
constraint e'#ations. "hey are represented as follows:

,
_

,
_

,
_

,
_

,
_

.n 21 1 .
n 2 22 21
n 1 12 11
n
2
1
n
2
1
n
2
1
a a a
a a a
a a a
'
0
0
0
0
(
(
(
(
c
c
c
&
x
x
x
$


, , , ,
1-&-4
Let $s = the col#mn vector of basic variables, &s
%
= the row vector of costs corresponding to $s, and 4 = the basis matri%
corresponding to $s.
(tep 1: $ntering +ector 6k:
Ior every nonbasic vector 6j, calc#late the coefficient
!j M cj = /6j M cj ma%imi7ation program!, or cj M !j = cj M /6j minimi7ation program!, where / = &s
%
4
51
.
"he nonbasic vector 6j with the most negative coefficient becomes the entering vector $.+.!, 6k.
*f more than one candidate for $.+. e%ists, choose one.
(tep /: Ceparting +ector 6r:
a! @alc#late the c#rrent basic $
s
: $s = 4
51
(
b! @orresponding to the entering vector 6k, calc#late the constraint coefficients t): tk = 4
51
6k
c! @alc#late the ratio 7:
. 2 1 i 0 t
t
$
ik
ik
i s
i
, , , , ,
!
min

'

>
"he departing vector C.+.!, 6r, is the one that satisfies the above condition.
Dote: *f all tkj P 0, there is no bo#nded sol#tion for the problem. (top.
(tep .: Dew Qasis:
4
51
ne+ = 84
51
, where 8 = 21, J, 2r51, 9, 2r+1, J, 2.!
Dote,

'

,
_

r i if
t
1
r i if
t
t
+,ere
rk
rk
ik
.
2
1
,
,
,

and 2i is a col#mn vector with 1 in the ith element and 0 in the other . M 1! elements.
(et 4
51
= 4
51
ne+ and repeat steps 1 thro#gh ., #ntil the following optimality condition is satisfied.
!j M cj 0 ma%imi7ation problem!, or cj M !j 0 minimi7ation problem!
"hen the optimal sol#tion is as follows: $s = 4
51
(= ! = &s
%
$s
Solved Problems
1. :se revised simple% method to solve the following problem.
;a%imi7e: ! = 10x1 9 11x2
s.t. x1 + 2x2 150
3x1 + 4x2 200
6x1 + x2 175
with: x1 and x2 nonnegative
"his program is p#t in standard form by introd#cing the slac) variables x3, x4, and x5,
;a%imi7e: ! = 10x1 9 11x2 + 0x3 + 0x4 + 0x5
s.t. x1 + 2x2 9 x3 = 150
3x1 + 4x2 9 x4 = 200
6x1 + x2 9 x5 = 175
<ith all variables nonnegative
61 =

,
_

6
3
1
, 62 =

,
_

1
4
2
, 63 =

,
_

0
0
1
, 64 =

,
_

0
1
0
, 65 =

,
_

1
0
0
, ( =

,
_

175
200
150
:nitiali!ati-n;
$s = x3, x4, x5!
"
= &s
%
= 0, 0, 0!
4 = 63, 64, 65! = : = 4
51
=

,
_

1 0 0
0 1 0
0 0 1
:terati-n <1;
"he nonbasic vectors are 61 and 62.
a! $ntering +ector: / = &s
%
4
51
= 0, 0, 0!: = 0, 0, 0!
!1 M c1, !2 M c2! = /61, 62! M c1, c2! = 0, 0, 0!

,
_

1 6
4 3
2 1
M 10, 11! = M10, M11!
12&-4
(ince the most negative coefficient corresponds to 62, it becomes the entering vector $.+.!
b! Ceparting +ector:
$s = 4
51
( = :( = ( =

,
_

175
200
150
t2 = 4
&1
62 = :62 = 62 =

,
_

1
4
2
50
1
175
4
200
2
150

'

, , min
(ince the minim#m ratio corresponds to 64, it becomes the departing vector C.+.!
c! Dew Qasis:

,
_

,
_

,
_

4 1
4 1
2 1
4 1
4 1
4 2
t
t
t
1
t
t
42
52
42
42
32
= $ = 21, 9, 23!

,
_



1 , 1 4
4 , 1 4
4 / 1 1
1 1
8 8: 84 4
ne+
(#mmary of *teration R1:
$s = x3, x2, x5!
%
= &s
%
= 0, 11, 0!
:terati-n <2;
Dow the nonbasic vectors are 61 and 64
a! $ntering +ector:
/ = &s
%
4
51
=
( ) ( ) 0 4 11 0
1 4 1 0
0 4 1 0
0 2 1 1
0 11 0 , , , ,

,
_

!1 M c1, !4 M c4! = /61, 64! M c1, c4! = 0, 11)4 , 0!

,
_

0 6
1 3
0 1
M 10, 0! = M15,, 11)4!
(ince the most negative coefficient corresponds to 61, it becomes the entering vector $.+.!
b! Ceparting +ector:

,
_

,
_

,
_



1/-
-4
-4
11-
/44
1-4
1 , 1 4
4 , 1 4
4 / 1 1
1
( 4 $
s

,
_

,
_

,
_



, /1
, .
/ 1
2
.
1
1 , 1 4
4 , 1 4
4 / 1 1
1
1
1
6 4 t
/1 -44
, /1
1/-
,
, .
-4
, min

'


(ince the minim#m ratio corresponds to 65, it becomes the departing vector C.+.!.
11&-4
c! Dew Qasis:

,
_

,
_

,
_

/1 ,
1 1
/1 /
, /1
1
, /1
, .
, /1
/ 1
1
-1
-1
/1
-/
.1
t
t
t
t
t
= $ = 21, 22, 9!

,
_

,
_

,
_



21 4 21 1 0
7 1 7 2 0
21 2 21 11 1
1 4 1 0
0 4 1 0
0 2 1 1
21 4 0 0
7 1 1 0
21 2 0 1
84 4
1 1
ne+
(#mmary of *teration R/:
$s = x3, x2, x1!
%
= &s
%
= 0, 11, 10!
:terati-n <3;
Dow the nonbasic vectors are 65 and 64.
a! $ntering +ector:
( ) ( ) . 1 , . A , 4
/1 , /1 1 4
1 1 1 / 4
/1 / /1 11 1
14 , 11 , 4
1

,
_



4 & /
%
s
!5 M c5, !4 M c4! = /65, 64! M c5, c4! = 0, 8)3 , 1)3!

,
_

4 1
1 4
4 4
M 0, 0! = 153, 8)3!
(ince all the coefficients are nonnegative, the above step gives the optimal basis. "he optimal val#es of the variables and the
obEective f#nction are as follows:

,
_

,
_

,
_

,
_

/1 -44
1 //-
/1 1.44
11-
/44
1-4
/1 , /1 1 4
1 1 1 / 4
/1 / /1 11 1
1
1
/
.
( 4
x
x
x
( ) . 111-
/1 -44
1 //-
/1 1.44
14 , 11 , 4

,
_


s
%
s
$ & !
/. :se the revised simple% method to solve the following problem.
;inimi7e: ! = 3x1 + 2x2 + 4x3 + 6x4
s.t. x1 + 2x2 + x3 + x4 1000
2x1 + x2 + 3x3 + 7x4 1500
with: all variables nonnegative
"his program is p#t in standard form by introd#cing the s#rpl#s variables x5 and x7, and the artificial variables x6 and x8.
;inimi7e: ! = 3x1 + 2x2 + 4x3 +6x4 + 0x5 + x6 + 0x7 + x8
s.t. x1 + 2x2 + x3 + x4 x5 + x6 = 1000
2x1 + x2 + 3x3 + 7x4 x7 + x8 = 1500
with: all variables nonnegative

,
_

,
_

,
_

,
_

,
_

,
_

,
_

,
_

,
_

1500
1000
(
1
0
6
1
0
6
0
1
6
0
1
6
7
1
6
3
1
6
1
2
6
2
1
6
8 7 6 5 4 3 2 1
, , , , , , , ,
:nitiali!ati-n;
$s = =x6, x8>
%
? &s
%
= =, >
( )

,
_



1 0
0 1
4 : 6 6 4
1
8 6
,
:terati-n <1;
"he nonbasic vectors are 61, 62, 63, 64, 65, and 67.
a! $ntering +ector:
1A&-4
( ) ( ) : 4 & /
1 %
s
, ,

( ) ( ) ( )
7 5 4 3 2 1 7 5 4 3 2 1 7 7 5 5 4 4 3 3 2 2 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )

,
_


1 0 7 3 1 2
0 1 1 1 2 1
0 0 6 4 2 3 , , , , , ,
( ) 6 8 4 4 2 3 3 3 , , , , , + + + +
(ince the most negative coefficient corresponds to 64, it becomes the entering vector $.+.!.
b! Ceparting +ector:

,
_



1500
1000
( :( ( 4 $
1
s

,
_



7
1
6 :6 6 4 t
4 4 4
1
4
{ } 7 1500 7 1500 1000 , min
(ince the minim#m ratio corresponds to 68, it becomes the departing vector C.+!
c! Dew Qasis:
=

,
_

,
_

7 1
7 1
t
1
t
t
84
84
64
8 = =21, 9>

,
_




7 1 0
7 1 1
8 8: 84 4
1 1
ne+
(#mmary of *teration R1:
( ) ( ) 6 & x x $
%
s
%
4 6 s
, = ,
:terati-n <2;
Dow the nonbasic vectors are 61, 62, 63, 68, 65, and 67.
a! $ntering +ector:
( ) ( ) 7 7 6
7 1 0
7 1 1
6 4 & /
1 %
s

,
_




, ,
( ) ( ) ( )
7 5 8 3 2 1 7 5 8 3 2 1 7 7 5 5 8 8 3 3 2 2 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )

,
_


1 0 1 3 1 2
0 1 0 1 2 1
7 7 6 0 0 4 2 3 , , , , , ,
( ) 7 6 7 7 6 7 8 7 10 7 4 7 8 7 13 7 " 7 5 + + + + , , , , ,
(ince the most negative coefficient corresponds to 62, it becomes the entering vector $.+.!.
b! Ceparting +ector:
( ) ( ) 7 1500 7 5500 7 1500 7 1500 1000
1500
1000
7 1 0
7 1 1
( 4 $
1
s
, ,

,
_

,
_

,
_

,
_

,
_




7 1
7 13
1
2
7 1 0
7 1 1
6 4 t
2
1
2
3
5500
1500
3
5500
7 1
7 1500
7 13
7 5500

'

'

, min , min
(ince the minim#m ratio corresponds to 66, it becomes the departing vector C.+.!
c! Dew Qasis:
( )
2
62
42
62
2 8
13 1
13 7
7 13
7 1
7 13
1
t
t
t
1
, =

,
_

,
_

,
_

,
_

,
_

,
_



13 2 13 1
13 1 13 7
7 1 0
7 1 1
1 13 1
0 13 7
84 4
1 1
ne+
13&-4
(#mmary of *teration R/:
( ) ( ) 6 2 & x x $
%
s
%
4 2 s
, = ,
:terati-n <3;
Dow the nonbasic vectors are 61, 66, 63, 68, 65, and 67.
a! $ntering +ector:
( ) ( ) 13 10 13 8
13 2 13 1
13 1 13 7
6 2 4 & /
1 %
s
, ,

,
_



( ) ( ) ( )
7 5 8 3 6 1 7 5 8 3 6 1 7 7 5 5 8 8 3 3 6 6 1 1
6 6 6 6 6 6 / c c c c c c ! c ! c ! c ! c ! c ! c , , , , , , , , , , , , , , ,
( ) ( )

,
_


1 0 1 3 0 2
0 1 0 1 1 1
13 10 13 8 0 0 4 3 , , , , , ,
( ) 13 10 13 8 13 10 13 14 13 8 13 11 , , , , , + +
(ince all the coefficients are nonnegative, the above step gives the optimal basis. "he optimal val#es of the variables and the
obEective f#nction are as follows:

,
_

,
_

,
_

,
_


13 2000
13 5500
1500
1000
13 2 13 1
13 1 13 7
( 4
x
x
1
4
2
( ) 13 23000
13 2000
13 5500
6 2 $ & !
s
%
s

,
_

,
2armarkar3s #lgorit"m
"he simple% method of linear programming finds the optim#m sol#tion by starting at the origin and moving along the adEacent
corner points of the feasible sol#tions space. Altho#gh this techni'#e has been s#ccessf#l is solving LN problems of vario#s si7es,
the n#mber of iterations becomes prohibitive for some h#ge problems. "his is an e%ponential time algorithm.
?n the other hand, Sarmar)arBs interior point Lp algorithm is a polynomial time algorithm. "his new approach finds the optim#m
sol#tion by starting at a trial sol#tion and shooting thro#gh the interior of the feasible sol#tion space. Altho#gh this proEective
algorithm may be advantageo#s in solving very large LN problems, it becomes very c#mbersome for not&so&large problems.
@onsider the following form of the linear programming problem:
;inimi7e: ! = &
%
$
s.t. '$ = 0
1$ = 1
$ 0
where $ is a col#mn vector of si7e n= & is an integer col#mn vector of si7e n= 1 is a #nit row vector of si7e n= ' is an integer matri%
of si7e . T n!= n /.
*n addition, ass#me the following two conditions:
1. $0 = 15n, ..., 15n! is a feasible sol#tion.
/. minim#m ! = 0
Summary o$ 2armarkar3s Iterative Procedure
1. Nreliminary (tep:
k = 4
$0 = 15n, ..., 15n!
%
! 1
1

n n
r
n n . ! 1
/. *teration k:
a! Cefine the following:
i! @0 = $0
ii! Ak = diagU$kV, which is the diagonal matri% whose diagonal consists of the elements of $k.
iii!

,
_

1
k
'A
6
iv!
k
%
A & &
b! @omp#te the following:
i!
% % %
3
& 6 66 6 : & H ! G
1

Dote: *f &3 = 0, any feasible sol#tion becomes an optimal sol#tion. (top.
ii!
3
3
ne+
&
&
r @ @
4
iii! ! 1 !
1 ne+ k ne+ k k
@ A @ A $
+
iv! ! = &
%
$k+1
/4&-4
v! k = k 9 1
vi! Kepeat iteration k #ntil the obEective f#nction !! val#e is less than a prescribed tolerance B.
!rans$orming a )inear Programming Problem into 2armarkar3s Special Form
@onsider the following linear program in the matri% form:
;inimi7e: ! = &
%
$
s.t. '$ (
with $ 0
"he simplified steps of converting the above problem into Sarmar)arBs special form are as follows:
1. (ince every primal linear program in the variables x1, x2, C,xn has associated with it a d#al linear program in the variables +1,
+2, C, +. where . is the n#mber of constraints in the original program!, write the d#al of the given primal problem as shown
below:
;a%imi7e: ! = (
%
/
s.t. '
%
/ &
with / 0
/. a! *ntrod#ce slac) and s#rpl#s variables to the primal and d#al problems.
b! @ombine primal and d#al problems.
.. a! *ntrod#ce a bo#nding constraint

+ D + x
i i
D sho#ld be s#fficiently large to incl#de all feasible sol#tions of the original problem.!
b! *ntrod#ce a slac) s in the bo#nding constraint and obtain the following e'#ation:

+ + D s + x
i i
,. a! *ntrod#ce a d#mmy variable 1 s#bEect to the condition 1 = 1! to homogeni7e the constraints.
b! Keplace the e'#ations

+ + D s + x
i i
and 1 = 1
with the following e'#ivalent e'#ations:

+ + 4 D1 s + x
i i
and

+ + + + ! 1 D 1 s + x
i i
-. *ntrod#ce the following transformations so as to obtain one on the K6( of the last e'#ation:
xj = D 9 1!0j, j = 1, 2, ..., . 9 n
+j = D 9 1!0.9n9j, j = 1, 2, ..., . 9 n
s = D 9 1!02.+2n+1
1 = D 9 1!02.+2n+2
2. *ntrod#ce an artificial variable 02.+2n+2 to be minimi7ed! in all the e'#ations s#ch that the s#m of the coefficients in each
homogeneo#s e'#ation is 7ero and the coefficient of the artificial variable in the last e'#ation is one.
(olved Nroblems
1. @arry o#t the first two iterations of Sarmar)arBs algorithm for the following problem.
;inimi7e: ! = 2x2 x3
s.t. x1 2x2 + x3 = 0
x1 + x2 + x3 = 1
with all variables nonnegative
Nreliminary (tep:
k = 0
$0 = =1)n, C, 1)n>
%
= =1)3, 1)3, 1)3>
%
( ) ( ) 6 1 1 3 3 1 1 n n 1 r
( ) ( ) ( ) " 2 3 3 1 3 n 3 1 n
*teration 4:
@0 = $0 = =1)3, 1)3, 1)3>
%
A0 = 1iaEF$0G = 1iaEF1)3, 1)3, 1)3G
' = =1, 2, 1>? &
%
= =0, 2, 1>
'A0 = =1, 2, 1> 1iaEF1)3, 1)3, 1)3G = F1)3, 2)3, 1)3G

,
_

,
_

1 1 1
3 1 3 2 3 1
1
'A
6
0
( ) { } ( ) 3 1 3 2 0 3 1 3 1 3 1 1iaE 1 2 0 A & &
0
%
, , , , , ,
/1&-4
( )

,
_

,
_


333 0 0
0 5 1
66
3 0
0 667 0
66
1
% %
.
.
=
.
( )

,
_

5 0 0 5 0
0 1 0
5 0 0 5 0
6 66 6
1
% %
. .
. .
( ) ( )
% %
1
% %
6
167 0 0 167 0 & 6 66 6 : & . , , .
1
]
1



( ) ( ) 2362 0 167 0 0 167 0 &
2 2
6
. . . + +
( )
( )
( ) ( )
% % %
6
6
0 ne+
3"74 0 3333 0 26"2 0 167 0 0 167 0
2362 0
6 1 " 2
3 1 3 1 3 1
&
&
r @ @ . , . , . . , , .
.
, ,
( )
%
ne+ 1
3"74 0 3333 0 26"2 0 @ $ . , . , .
( )( ) 26"2 0 3"74 0 3333 0 26"2 0 1 2 0 $ & !
%
1
%
. . , . , . , ,
k = 0 + 1 = 1
*teration 1:
{ } { } 3"74 0 3333 0 26"2 0 1iaE $ 1iaE A
1 1
. , . , .
( ) { } ( ) 3"74 0 6666 0 0 3"74 0 3333 0 26"2 0 1iaE 1 2 0 &A &
1
. , . , . , . , . , ,
( ) { } ( ) 3"74 0 6666 0 26"2 0 3"74 0 3333 0 26"2 0 1iaE 1 2 1 'A
1
. , . , . . , . , . , ,

,
_

,
_

1 1 1
3"74 0 6666 0 26"2 0
1
'A
6
1
. . .
( )

,
_

,
_


333 0 0
0 482 1
66
3 0
0 675 0
66
1
% %
.
.
=
.
( )

,
_

567 0 05" 0 4"2 0


05" 0 ""2 0 067 0
4"2 0 067 0 441 0
6 66 6
1
% %
. . .
. . .
. . .
( ) ( )
% %
1
% %
6
132 0 018 0 151 0 & 6 66 6 : & . , . , .
1
]
1



( ) ( ) ( ) 2014 0 132 0 018 0 151 0 &
2 2 2
6
. . . . + +
( )
( )
( ) ( )
% % %
6
6
0 ne+
3"28 0 3414 0 2653 0 132 0 018 0 151 0
2014 0
6 1 " 2
3 1 3 1 3 1
&
&
r @ @ . , . , . . , . , .
.
, ,
{ }( ) ( )
% %
ne+ 1
1561 0 1138 0 0714 0 3"28 0 3414 0 2653 0 3"74 0 3333 0 26"2 0 1iaE @ A . , . , . . , . , . . , . , .
3413 0 @ A 1
ne+ 1
.
( )
%
ne+ 1
ne+ 1
2
4574 0 3334 0 20"2 0
@ A 1
@ A
$ . , . , .
( )( ) 20"4 0 4574 0 3334 0 20"2 0 1 2 0 $ & !
%
2
%
. . , . , . , ,
/. @arry o#t the first two iterations of Sarmar)arBs algorithm for the following problem.
;inimi7e: ! = 2x1 + x2 + 2x3 2x4
s.t. 2x1 + x2 + 2x3 2x4 3x5 = 0
2x1 x3 + x4 2x5 = 0
x1 + x2 + x3 + x4 + x5 = 0
with: all variables nonnegative
Nreliminary (tep:
k = 0
$0 = =1)n, C, 1)n>% = =1)5, 1)5, 1)5, 1)5, 1)5>
%
( ) ( ) 20 1 1 5 5 1 1 n n 1 r
( ) ( ) ( ) 15 4 3 5 1 5 n 3 1 n
*teration 4:
@0 = $0 = =1)n, C, 1)n>% = =1)5, 1)5, 1)5, 1)5, 1)5>
%
//&-4
A0 = 1iaEU$0V = 1iaEU1)5, 1)5, 1)5, 1)5, 1)5V
( ) 0 2 2 1 2 &
2 1 1 0 2
3 2 2 1 2
'
%
, , , , =

,
_

{ }

,
_

,
_

4 0 4 0 2 0 0 4 0
6 0 4 0 4 0 2 0 4 0
5 1 5 1 5 1 5 1 5 1 1iaE
2 1 1 0 2
3 2 2 1 2
'A
0
. . . .
. . . . .
, , , ,

,
_

,
_

1 1 1 1 1
4 0 2 0 2 0 0 4 0
6 0 4 0 4 0 2 0 4 0
1
'A
6
0
. . . .
. . . . .
( ) { } ( ) 0 4 0 4 0 2 0 4 0 5 1 5 1 5 1 5 1 5 1 1iaE 0 2 2 1 2 A & &
0
%
, . , . , . , . , , , , , , , ,
( )

,
_

,
_


2 0 0 0
0 "8"1 2 8152 0
0 8152 0 3587 1
66
5 0 0
0 4 0 24 0
0 24 0 88 0
66
1
% %
.
. .
. .
= . .
. .
( )

,
_

7761 0 2543 0 1457 0 1022 0 2783 0


2543 0 6674 0 2674 0 0587 0 2870 0
1457 0 2674 0 6674 0 3413 0 1130 0
1022 0 0587 0 3413 0 2543 0 2435 0
2783 0 2870 0 1130 0 2435 0 6348 0
6 66 6
1
% %
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
( ) ( )
% %
1
% %
3
1343 0 1526 0 0874 0 0613 0 167 0 & 6 66 6 : & . , . , . , . , .
1
]
1



( ) ( ) ( ) ( ) ( ) 283" 0 1343 0 1526 0 0874 0 0613 0 167 0 &
2 2 2 2 2
3
. . . . . . + + + +
( )
( )( )
( )
% %
3
3
0 ne+
1343 0 1526 0 0874 0 0613 0 167 0
283" 0
20 1 15 4
5 1 5 1 5 1 5 1 5 1
&
&
r @ @ . , . , . , . , .
.
, , , ,
( )
%
1718 0 2321 0 2184 0 212" 0 164" 0 . , . , . , . , .
( )
%
ne+ 1
1718 0 2321 0 2184 0 212" 0 164" 0 @ $ . , . , . , . , .
( )( ) 5154 0 1718 0 2321 0 2184 0 212" 0 164" 0 0 2 2 1 2 $ & !
%
1
%
. . , . , . , . , . , , , ,
k = 0 + 1 = 1
*teration 1:
{ } { } 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE $ 1iaE A
1 1
. , . , . , . , .
( ) { } ( ) 0 4642 0 4368 0 212" 0 32"8 0 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE 0 2 2 1 2 A & &
1
%
, . , . , . , . . , . , . , . , . , , , ,
{ }

,
_


1718 0 2321 0 2184 0 212" 0 164" 0 1iaE
2 1 1 0 2
3 2 2 1 2
'A
1
. , . , . , . , .

,
_

3436 0 2321 0 2184 0 0 32"8 0


5154 0 4642 0 4368 0 212" 0 32"8 0
. . . .
. . . . .

,
_

,
_

1 1 1 1 1
3436 0 2321 0 2184 0 0 32"8 0
5154 0 4642 0 4368 0 212" 0 32"8 0
1
'A
6
1
. . . .
. . . . .
( )

,
_

,
_


2 0 0 0
0 123" 3 3128 0
0 3128 0 242 1
66
5 0 0
0 3284 0 0827 0
0 0827 0 826 0
66
1
% %
.
. .
. .
= . .
. .
( )

,
_

787" 0 2355 0 1665 0 0866 0 2765 0


2355 0 7033 0 2736 0 0618 0 2730 0
1665 0 2736 0 6456 0 3300 0 1314 0
0866 0 0618 0 3300 0 2563 0 2653 0
2765 0 2730 0 1314 0 2653 0 606" 0
6 66 6
1
% %
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
( ) ( )
% %
1
% %
3
10"3 0 1214 0 0858 0 0446 0 1425 0 & 6 66 6 : & . , . , . , . , .
1
]
1



( ) ( ) ( ) ( ) ( ) 2374 0 10"3 0 1214 0 0858 0 0446 0 1425 0 &
2 2 2 2 2
3
. . . . . . + + + +
/.&-4
( )
( )( )
( )
% %
3
3
0 ne+
10"3 0 1214 0 0858 0 0446 0 1425 0
2374 0
20 1 15 4
5 1 5 1 5 1 5 1 5 1
&
&
r @ @ . , . , . , . , .
.
, , , ,
( )
%
1725 0 2305 0 2216 0 2112 0 1642 0 . , . , . , . , .
{ }( )
%
ne+ 1
1725 0 2305 0 2216 0 2112 0 1642 0 1718 0 2321 0 2184 0 212" 0 164" 0 1iaE @ A . , . , . , . , . . , . , . , . , .
( )
%
02"6 0 0535 0 0484 0 0450 0 0271 0 . , . , . , . , .
2036 0 @ A 1
ne+ 1
.
( )
%
ne+ 1
ne+ 1
2
1456 0 2628 0 2377 0 220" 0 1330 0
@ A 1
@ A
$ . , . , . , . , .
( )( ) 4367 0 1456 0 2628 0 2377 0 220" 0 1330 0 0 2 2 1 2 $ & !
%
2
%
. . , . , . , . , . , , , ,
.. @onvert the following problem into Sarmar)arBs special from:
;a%imi7e: ! = 2x1 + x2
s.t. x1 + x2 5
x1 x2 3
with: all variables nonnegative
i! C#al of the given problem.
;inimi7e: ! = 5+1 + 3+2
s.t. +1 + +2 2
+1 +2 1
with: all variables nonnegative
ii! *ntrod#ction of slac) and s#rpl#s variables and combination of primal and d#al problems.
x1 + x2 + x3 = 5 +1 + +2 +3 = 2
x1 x2 + x4 = 3 +1 +2 +4 = 1
2x1 + x2 = 5+1 + 3+2
with: all variables nonnegative
iii! Addition of a bo#ndary constraint with slac) variable s.
x1 + x2 + x3 = 5 +1 + +2 +3 = 2
x1 x2 + x4 = 3 +1 +2 +4 = 1
2x1 + x2 5+1 3+2 = 0



+ +
4
1 i
4
1 i
i i
D s + x
with: all variables nonnegative
iv! 6omogeni7ed e'#ivalent system with d#mmy variable 1.
x1 + x2 + x3 51 = 0 +1 + +2 +3 21 = 0
x1 x2 + x4 31 = 0 +1 +2 +4 1 = 0
2x1 + x2 5+1 3+2 = 0



+ +
4
1 i
4
1 i
i i
0 D1 s + x

( )


+ + + +
4
1 i
4
1 i
i i
1 D 1 s + x
with: all variables nonnegative
v! *ntrod#ction of the transformations.
xj = =D + 1>0j, j = 1, C, 4 s = =D + 1>"
+j = =D + 1>04+j, j = 1, C, 4 1 = =D + 1>10
"he above transformations yield the following system:
01 + 02 + 03 5010 = 0 05 + 06 07 2010 = 0
01 02 + 04 3010 = 0 05 06 08 010 = 0
201 + 02 505 306 = 0

0 D0 0
10
"
1 i
i


1 0
10
1 i
i

with: all variables nonnegative


vi! *ntrod#ction of the artificial variable with appropriate constraint coefficients.
;inimi7e: 011
s.t. 01 + 02 + 03 50" + 2011 = 0
01 02 + 04 3010 + 2011 = 0
/,&-4
05 + 06 07 2010 + 011 = 0
05 06 08 010 + 2011 = 0
201 + 02 505 306 + 5011 = 0
( )

+
"
1 i
11 10 i
0 0 " D D0 0

11
1 i
i
1 0
with: all variables nonnegative
,. @onvert the following problem into Sarmar)arBs special from:
;a%imi7e: ! = x1 + 4x2
s.t. x1 + 2x2 10
2x1 + 3x2 20
x1 6
with: all variables nonnegative
i! C#al of the give problem.
;inimi7e: ! = 10+1 + 20+2 + 6+3
s.t. +1 + 2+2 + +3 1
2+1 + 3+2 4
with: all variables nonnegative
ii! *ntrod#ction of slac) and s#rpl#s variables and combination of primal and d#al problems.
x1 + 2x2 + x3 = 10 +1 + 2+2 + +3 +4 = 1
2x1 + 3x2 + x4 = 20 2+1 + 3+2 +5 = 4
x1 + x5 = 6
x1 + 4x2 = 10+1 + 20+2 + 6+3
with: all variables nonnegative
iii! Addition of a bo#ndary constraint with slac) variable s.
x1 + 2x2 + x3 = 10 +1 + 2+2 + +3 +4 = 1
2x1 + 3x2 + x4 = 20 2+1 + 3+2 +5 = 4
x1 + x5 = 6
x1 + 4x2 10+1 20+2 6+3 = 0

+ +
5
i
i
5
i
i
D s + x
with: all variables nonnegative
iv! 6omogeni7ed e'#ivalent system with d#mmy variable 1.
x1 + 2x2 + x3 101 = 0 +1 + 2+2 + +3 +4 1 = 0
2x1 + 3x2 + x4 201 = 0 2+1 + 3+2 +5 41 = 0
x1 + x5 61 = 0
x1 + 4x2 10+1 20+2 6+3 = 0

+ +
5
i
i
5
i
i
0 D1 s + x
( )

+ + + +
5
i
i
5
i
i
1 D 1 s + x
with: all variables nonnegative
v! *ntrod#ction of the transformations
xj = =D + 1>0j, j = 1, C, 5 s = =D + 1>11
+j = =D + 1>05+j, j = 1, C, 5 1 = =D + 1>12
"he following transformations yield the following system:
01 + 202 + 03 10012 = 0 06 + 207 + 08 0" 012 = 0
201 + 302 + 04 20012 = 0 206 + 307 010 4012 = 0
01 + 05 6012 = 0
01 + 402 1006 2007 608 = 0
0 D0 0
12
11
i
i

1 0
12
i
i

vi! *ntrod#ction of the artificial variable with appropriate constraint coefficients.


;inimi7e: 013
s.t.: 01 + 202 + 03 10012 + 6013 = 0
201 + 302 + 04 20012 + 14013 = 0
/-&-4
01 + 05 6012 + 4013 = 0
06 + 207 + 08 0" 012 2013 = 0
206 + 307 010 4012 = 0
01 + 402 1006 2007 608 + 31013 = 0
( ) 0 0 11 D D0 0
13 12
11
i
i
+

1 0
13
i
i

with: all variables nonnegative


/2&-4
Integer Programming
4ranc"5and54ound #lgorit"m
4ranc"ing
"he first a33r-xi.ati-n to the sol#tion of any integer program may be obtained by ignoring the integer re'#irement and solving
the res#lting linear program by one of the techni'#es already presented. *f the optimal sol#tion to the linear program is not an
integral, components of the first appro%imation can be ro#nd to the nearest feasible integers M sec-n1 a33r-xi.ati-n.
*f the first appro%imation contains a variable that is not integral, say xj, then i1 H xj H i2, where i1 and i2 are consec#tive,
nonnegative integers. "wo new integer programs are then created by a#gmenting the original integer program with either the
constraint xj i1 or the constraint xj i2. "his process called branching, has the effect of shrin)ing the feasible region in a way that
eliminates from f#rther consideration the c#rrent nonintegral sol#tion for xj b#t still preserves all possible integral sol#tions to the
original problem.
$%ample 1
ma%imi7e: ! = 10x1 + x2
s.t. 2x1 + 5x2 11 1!
with x1 and x2 nonnegative and integral
we consider the associated linear program obtained by deleting the integral re'#irement. "he sol#tion is fo#nd to be x1 = 5*5, x2 =
0, with ! = 55. (ince 5 H x1 H 6, branching creates the two new integer programs
ma%imi7e: ! = 10x1 + x2
s.t. 2x1 + 5x2 11 /!
x1 5
with x1 and x2 nonnegative and integral
ma%imi7e: ! = 10x1 + x2
s.t. 2x1 + 5x2 11 .!
x1 6
with x1 and x2 nonnegative and integral
Ior the two integer programs created by the branching process, first appro%imations are obtained by again ignoring the integer
re'#irement and solving the res#lting linear programs. *f either first appro%imation is still nonintegral, then the integer program
which gave rise to that first appro%imation becomes a candidate for f#rther branching.
$%ample/
Qy solving ine'#alities we find that /! has the first appro%imation x1 = 5, x2 = 0*2, with ! = 50*2, while program .! has no
feasible sol#tion. "h#s, program /! is a candidate for f#rther branching. (ince 0 H x2 H 1, we a#gment /! with either x2 0 or x2
1, and obtain the two new programs
ma%imi7e: ! = 10x1 + x2
s.t. 2x1 + 5x2 11 ,!
x1 5
x2 0
with x1 and x2 nonnegative and integral
ma%imi7e: ! = 10x1 + x2
s.t. 2x1 + 5x2 11 -!
x1 5
x2 1
with x1 and x2 nonnegative and integral
<ith the integer re'#irements ignored, the sol#tion to program ,! is x1 = 5, x2 = 0, with ! = 50, while the sol#tion to program -!
is x1 = 3, x2 = 1, with ! = 31. (ince both these first appro%imations are integral, no f#rther branching is re'#ired.
4ounding
Ass#me that the obEective f#nction is to be ma%imi7ed. Qranching contin#es #ntil an integral first appro%imation which is th#s an
integral sol#tion! is obtained. "he val#e of the obEective for this first integral sol#tion becomes a lower bo#nd for the problem, and
all programs whose first appro%imations, integral or not, yield val#es of the obEective f#nction smaller than the lower bo#nd are
discarded. if, in the process, a new integral sol#tion is #ncovered having a val#e of the obEective f#nction greater than the c#rrent
lower bo#nd, then this val#e of the obEective f#nction becomes the new lower bo#nd. "he branching process contin#es #ntil there
are no programs with nonintegral first appro%imations remaining #nder consideration. At this point, the c#rrent lower bo#nd
sol#tion is the optimal sol#tion to the original integer program.
*f the obEective f#nction is to be minimi7ed, the proced#re remains the same, e%cept that #pper bo#nds are #sed.
Cut #lgorit"ms
At each stage of branching in the branch&and&bo#nd algorithm the c#rrent feasible region is c#t into two smaller regions by the
imposition of two new constraints derived from the first appro%imation to the c#rrent program. "his splitting is s#ch that the
optimal sol#tion to the c#rrent program m#st show #p as the optimal sol#tion to one of the tow new programs. "he c#t algorithms
operate essentially in li)e fashion, the only difference being that a single new constraint is added at each stage, whereby the
feasible region is diminished witho#t being split.
/1&-4
!"e Gomory #lgorit"m
"he new constraint are determined by the following three&step proced#re
(tep1: *n the c#rrent final simple% tablea#, select one any one! of the nonintegral variables and, witho#t assigning 7ero val#es to
the nonbasic variables, consider the constraint e'#ation represented by the row of the selected variable.
/ /- , . 1 4 4 ,
1 , 1 1 4 1 / 1
/ 11 / 1 . 1 1 4 / 1
/
.
- , . / 1


x
x
x x x x x
"he noninteger assignment for %. came from the first row of the tablea#, which represents the constraint
/
11
/
1
.
1
/
1
/ , . 1
+ + x x x x
(tep/: Kewrite each fractional coefficient and constant in the constraint e'#ation obtained form (tep1 as the s#m of an integer and
a positive fraction between 4 and 1. "hen rewrite the e'#ation so that the left&hand side contains only terms with fractional
coefficients and a fractional constant!, while the right&hand side contains only terms with integral coefficients and an
integral constant!. $'#ation becomes
/
1
-
/
1
4
.
/
.
/
1
1
- , . 1
+

,
_

+ +

,
_

+ + +

,
_

+ x x x x
or
, . 1 - , 1
. -
/
1
/
1
.
/
/
1
x x x x x x + + + +
(tep.: Ke'#ire the left&hand side of the rewritten e'#ation to be nonnegative. "he res#lting ine'#ality is the new constraint.
4
/
1
/
1
.
/
/
1
- , 1
+ + x x x or
/
1
/
1
.
/
/
1
/ , 1
+ + x x x
Solved Problem
ma%imi7e: ! = 2x1 + x2
s.t. 2x1 + 5x2 17 1!
3x1 + 2x2 10
with: x1 and x2 nonnegative and integral
*gnoring the integer re'#irements and applying the simple% method to the res#lting linear program, we obtain the optimal tablea#
after one iteration.
. /4 . / 4 . 1 4
. 14 . 1 4 . / 1
. .1 . / 1 . 11 4
1
.
, . / 1
x
x
x x x x

"ablea# 1
"he first appro%imation to program, therefore, is x1 = 10)3, x3 = 31)3, x2 = x4 = 0. Qoth x1 and x3 are noninteger. Arbitrary
selecting x1, we consider the constraint represented by the second row of the tablea#, the row defining x1= namely,
.
14
.
1
.
/
, / 1
+ + x x x
<riting each fraction as the s#m of an integer and a fraction between 4 and 1, we have
.
1
.
.
1
4
.
/
4
, / 1
+

,
_

+ +

,
_

+ + x x x or
1 , /
.
.
1
.
1
.
/
x x x +
Ke'#iring the left&hand side of this e'#ation to be nonnegative, we obtain
4
.
1
.
1
.
/
, /
+ x x or 1 /
, /
+x x
as the new constraint. Kewriting the constraints of the original program in the forms s#ggested by the tablea# and adding the new
constraint, we generate the new program
ma%imi7e: ! = 2x1 + x2 + 0x3 + 0x4
s.t.
.
11
x2 + x3
.
/
x4 =
.
.1
/!
x1 +
.
/
x2 +
.
1
x4 =
.
14
2x2 + x4 1
with: all variables nonnegative and integral
/A&-4
A s#rpl#s variable, x5, and an artificial variable, x6, are introd#ced into the ine'#ality constraint of /! and then the two&phase
method is applied, with x1, x3, and x6 as the initial set of basic variables. "he optimal simple% tablea# is obtained after only one
iteration. "he first appro%imation to program /! is th#s x1 = 3, x2 = 1)2, x3 = 17)2, x4 = x5 = 0. @hoosing x2 to generate the new
constraint, we obtain from the third row of "ablea# /
/ 1. 2 1 / 1 4 4 4
/ 1 / 1 / 1 4 1 4
. . 1 4 4 4 1
/ 11 2 11 / - 1 4 4
/
1
.
- , . / 1

x
x
x
x x x x x
"ablea# /
4
/
1
/
1
/
1
- ,
+ x x or 1
- ,
+x x
"his, combined with the constraints of program /! in the forms s#ggested by the "ablea# /, gives the new integer program
ma%imi7e: ! = 2x1 + x2 + 0x3 + 0x4 + 0x5
s.t. x3
/
-
x4 +
/
-
x5 =
/
11
x1 +
.
1
x5 = 3 .!
x2 +
/
1
x4
/
1
x5=
/
1
x4 + x5 1
with: all variables nonnegative and integral
*gnoring the integer constraint and applying the two&phase method to program .!, with x1, x2, x3, and x7 artificial! as the initial
basic set, we obtain the optimal "ablea# ..
. 13 2 1 4 . 1 4 4 4
1 1 1 1 4 4 4
1 / 1 4 1 4 1 4
. A . 1 4 . 1 4 4 1
. /4 2 11 4 . 1. 1 4 4
-
/
1
.
2 - , . / 1

x
x
x
x
x x x x x x
"ablea# .
A new iteration of the process is started from x1 =
.
A
in "ablea# .. "his res#lts in a program whose sol#tion is integral, with x1 =
3, x2 = 0, and ! = 6. "his sol#tion is then the optimal sol#tion to integer program 1!.
Summary o$ t"e Gomory cut algorit"m
@onsider the optimal tablea# that res#lts from applying the simple% method to an integer program with the integer re'#irements
ignored, and ass#me that one of the basic variables, xb is nonintegral. "he constraint e'#ation corresponding to the tablea# row
that determined xb m#st have the form

+
4
0 x 0 x
j j b
1!
where the s#m is over all nonbasic variables. "he 0&terms are the coefficients and the constant term appearing in the tablea# row
determining xb. (ince xb is obtained from 1! by setting the nonbasic variables e'#al to 7ero, it follows that 00 is also nonintegral.
<rite each 0&term in 1! as the s#m of an integer and a nonnegative fraction less than 1:
j j j
f i 0 +
and
4 4 4
f i 0 +
(ome of the fj may be 7ero, b#t f0 is g#aranteed to be positive. $'#ation 1! becomes
( )

+ + +
4 4
f i x f i x
j j j b
or

+
j j j j b
x f f i x i x
4 4
/!
*f each %&variable is re'#ired to be integral, then the left&hand side of /! is integral, which forces the right&hand side also to be
integral. Q#t, since each fj and xj is nonnegative, so too is
j j
x f . "he right&hand side of /! then is an integer which is
smaller than a positive fraction less than 1= that is, a nonpositive integer.

4
4 j j
x f f or

4
4
f x f
j j
"his is the new constraint in the >omory algorithm.
/3&-4
#not"er cut algorit"m
@onsider 1! in the s#mmary of >omory algorithm. *f each nonbasic variable xj is 7ero, then xb = 00 is nonintegral. *f xb is to
become integer&val#ed, then at least one of the nonbasic xj m#st be made different from 7ero. (ince all variables are re'#ired to be
nonnegative and integral, it follows that at least one nonbasic variable m#st be made greater than or e'#al to 1. "his in t#rn
implies that the s#m of all the nonbasic variables m#st be made greater that or e'#al to 1. *f this condition is #sed as the new
constraint to be adEoined to the original integer program, we have the c#t algorithm first s#ggested by Can7ig.
Solved problem
ma%imi7e: ! = 3x1 + 4x2
s.t. 2x1 + x2 6
2x1 + 3x2 "
with: x1 and x2 nonnegative and integral
*ntrod#cing slac) variables x3 and x4 and then solving the res#lting program, with the integer re'#irements ignored, by the simple%
method, we obtain "ablea# 1.
1- . 1/ /- . 1 /- . 4 4 4
- . 1 - . 4 - . 4 1 4
/- . / /- . 4 1- . 4 4 1
/
1
, . / 1

x
x
x x x x
"ablea# 1
"he first appro%imation is, therefore, x1 = 2*25, x2 = 1*5, which is not integral. "he nonbasic variables are x3 and x4, so the new
constraint is x3 + x4 1. Appending this constraint to "ablea# 1, after the introd#ction of s#rpl#s variable x5 and artificial variable
x6, and solving the res#lting program by two&phase method, we generate "ablea# /.
/- . 1/ /- . 4 1 4 4 4
1 1 1 1 4 4
- - . 4 1 4 1 4
- . 1 1- . 4 1 4 4 1
.
/
1
- , . / 1

x
x
x
x x x x x
"ablea# /
*t follows from "ablea# / that x1 = 1*5, x2 = 2, x3 = 1, with x4 and x5 nonbasic. (ince this sol#tion is nonintegral, we ta)e x4 + x5
1 as the new constraint. AdEoining this constraint to "ablea# /, after the introd#ction of s#rpl#s variable x6 and artificial variable
x7, and solving the res#lting program by the two&phase method, we generate "ablea# ..
/- . 1/ /- . 4 4 1- . 4 4 4 4
1 1 1 1 4 4 4
/ 1 4 / 1 4 4
- . / - . 4 4 - . 1 4 1 4
1- . 4 1- . 4 4 1- . 1 4 4 1
-
.
/
1
2 - , . / 1

x
x
x
x
x x x x x x
"ablea# .
Irom "ablea# ., the c#rrent optimal sol#tion is nonintegral, with nonbasic variables x4 and x6. "he new constraint is th#s x4 + x6
1. AdEoining it to "ablea# . and solving the res#lting program by the two&phase method, we obtain x1 = 0, x2 = 3, with ! = 12.
(ince this sol#tion is integral, it is optimal sol#tion to the original integer program.
.4&-4
Integer Programming: !"e !ransportation #lgorit"m
A transportation problem involves . so#rces, each of which has available ai i = 1, 2, C, .! #nits of a homogeneo#s prod#ct, and
n destinations, each of which re'#ires bj j = 1, 2, C, n! #nits of this prod#ct. "he n#mbers ai and bj are positive integers. "he cost
cij of transporting one #nit of prod#ct from the ith so#rce to the jth destination is given for each i and j. "he obEective is to develop
an integral transportation sched#le the prod#ct may not be fractionali7ed! that meets all demands from c#rrent inventory at a
minim#m total shipping cost.
*t is ass#med that total s#pply and total demand are e'#al= that is

.
i
n
j
j i
b a
1 1
1!
$'#ation 1! is g#aranteed by creating either a fictitio#s destination with a demand e'#al to the s#rpl#s if total demand is less than
total s#pply, or a fictitio#s so#rce with a s#pply e'#al to the shortage if total demand e%ceeds total s#pply.
Let xij represent the #n)nown! n#mber of #nits to be shipped from so#rce i to destination j. "hen the standard mathematical
model for this problem is:
minimi7e:

.
i
n
j
ij ij
x c !
1 1
s.t.
( ) . i a x
n
j
i ij
, , 1
1

/!
( ) n j b x
.
i
i ij
, , 1
1

with: all xij nonnegative and integral


!"e !ransportation #lgorit"m
"he first appro%imation to system /! is always integral, and therefore is always the optimal sol#tion. Kather than determining this
first appro%imation by a direct application of the simple% method, we find it more efficient to wor) with "ablea# 1. All entries are
self&e%planatory, with the e%ception of the terms 2i, and vj, which will be e%plained shortly. "he transportation algorithm is the
simple% method speciali7ed to the format of "ablea# 1= as #s#al, it involves
i! finding an initial, basic feasible sol#tion=
ii! testing the sol#tion for optimality=
iii! improving the sol#tion when it is not optimal=
iv! repeating steps ii! and iii! #ntil the optimal sol#tion is obtained.
Cestinations
1 / . J n (#pply 2i
(
o
#
r
c
e
s
1
c11 c12 c13 c1n
a1 21 x11 x12 x13 C x1n
/
c21 c22 c23 c2n
a2 22 x21 x22 x23 C x2n
JJ CC CC CC C CC JJ JJ
.
c.
1
c.
2
c.
3
c.
n
a. 2.
x.
1
x.
2
x.
3 C
x.
n
Cemand b1 b2 b3 C bn
vj v1 v2 v3 C vn
"ablea# 1
#n Initial 4asic Solution
Nort"%est Corner 1ule. Qeginning with 1.1! cell in "ablea# 1 the northwest corner!, allocate to x11 as many #nits as possible
witho#t violating the constraints. "his will be the smaller of a1 and b1. "hereafter, contin#e by moving one cell to the right, if
some s#pply remains, or if not, one cell down. At each step, allocate as m#ch as possible to the cell variable! #nder consideration
witho#t violating the constraints: the s#m of the i5th row allocations cannot e%ceed ai, the s#m of the j&th col#mn allocations
cannot e%ceed bj, and no allocation can be negative. "he allocation may be 7ero.
Vogel3s et"od. Ior each row and each col#mn having some s#pply or some demand remaining calc#late its difference, which is
the nonnegative difference between the two smallest shipping costs cij associated with #nassigned variables in that row or col#mn.
@onsider the row or col#mn having the largest difference= in case of a tie, arbitrarily choose one. *n this row or col#mn, locate that
#nassigned variable cell! having the smallest #nit shipping cost and allocate to it as many #nits as possible witho#t violating the
constraints. Kecalc#late the new differences and repeat the above proced#re #ntil all demands are satisfied.
.1&-4
+ariables that are assigned val#es by either one of these starting proced#res become the basic variables in the initial sol#tion. "he
#nassigned variables are nonbasic and, therefore, 7ero. <e adopt the convention of not entering the nonbasic variables in "ablea#
1 M they are #nderstood to be 7ero M and of indicating basic&variable allocations in boldface type.
"he northwest corner r#le is the simpler of the two r#les to apply. 6owever, +ogelBs method, which ta)es into acco#nt the #nit
shipping costs, #s#ally res#lts in a closer&to&optimal starting sol#tion.
!est $or 6ptimality
Assign on any one! of the 2i or vi in "ablea# 1 the val#e 7ero and calc#late the remaining 2i and vi so that for each basic variable
2i + vj = cij. "hen, for each nonbasic variable, calc#late the '#antity cij 2i vj. *f all these letter '#antities are nonnegative, the
c#rrent sol#tion is optimal= otherwise, the c#rrent sol#tion is not optimal.
Improving t"e Solution
Cefinition: A loop is a se'#ence of cells in "ablea# 1 s#ch that: i! each pair of consec#tive cells lie in either the same row or the
same col#mn= ii! no three consec#tive cells lie in the same row or col#mn= iii! the first and last cells of the se'#ence lie in the
same row or col#mn iv! no cell appears more that once in the se'#ence.
Example &
"he se'#ences U1,/!, 1,,!, /,,!, /, 2!, ,, 2!, ,, /!V and U1, .!, 1, 2!, ., 2!, ., 1!, /, 1!, /, /!, ,, /!, ,, ,!, /, ,!, /, .!V
ill#strated if Iig 1 and /, respectively, are loops. Dote that a row or col#mn can have more than two cells in the loop as the
second row of Iig /!, b#t no more than two can be consec#tive.
@onsider the nonbasic variable corresponding to the most negative of the '#antities cij 2i vj calc#lated in the test for optimality=
it is made the incoming variable. @onstr#ct a loop consisting e%cl#sively of this incoming variable cell! and c#rrent basic
variables cells!. "hen allocate to the incoming cell as many #nits as possible s#ch that, after appropriate adE#stments have been
made to the other cells in the loop, the s#pply and demand constraint are not violated, all allocations remain nonnegative, and one
of the old basic variables has been red#ced to 7ero where#pon it ceases to be basic!.
(egeneracy
*n view of condition 1!, only n + . 1 of the constraint e'#ations in system /! are independent. "hen a nondegenerate basic
feasible sol#tion will be characteri7ed by positive val#es for e%actly n + . 1 basic variables. *f the process of improving the
c#rrent basic sol#tion res#lts in two or more c#rrent basic variables being red#ced to 7ero sim#ltaneo#sly, only one is allowed to
become nonbasic solverBs choice, altho#gh the variable with the largest #nit shipping cost is preferred!. "he other variables!
remains remain! basic, b#t with a 7ero allocation, thereby rendering the new basic sol#tion degenerate.
"he northwest corner r#le always generates an initial basic sol#tion= b#t it may fail to provide n + . 1 positive val#es, th#s
yielding a degenerate sol#tion. *f +ogelBs method is #sed, and does not yield that same n#mber of positive val#es, additional
variables with 7ero allocations m#st be designated as basic. "he choice is arbitrary, to a point: basic variables cannot form loops,
and preference is #s#ally given to variables with the lowest associated shipping costs.
*mproving a degenerate sol#tion may res#lt in replacing one basic variable having a 7ero val#e by another s#ch. Altho#gh the two
degenerate sol#tions are effectively the same M only the designation of the basic variables has changes, not their val#es M the
additional iteration is necessary for the transportation algorithm to proceed.
(olved Nroblems
1. A car rental company is faced with an allocation problem res#lting from rental agreements that allow cars to be ret#rned to
locations other than those at which they were originally rented. At the present time, there are two locations so#rces! with 1- and
1. s#rpl#s cars, respectively, and fo#r locations destinations! re'#iring 3, 2, 1, and 3 cars, respectively. :nit transportations costs
in dollars! between the locations are as follows:
Cest. 1 Cest. / Cest. . Cest. ,
(o#rce 1 ,- 11 /1 .4
(o#rce / 1, 1A 13 .1
(et #p the initial transportation tablea# for the minim#m&cost sched#le.
./&-4
(ince the total demand 3 9 2 9 1 9 3 = .1! e%ceeds the total s#pply 1- 9 1. = /A!, a d#mmy so#rce is created having a s#pply
e'#al to the .&#nit shortage. *n reality, shipments from this fictitio#s so#rce are never made, so the associated shipping costs are
ta)en as 7ero. Nositive allocations from this so#rce to a destination represent cars that cannot be delivered d#e to a shortage of
s#pply= they are shortages a destination will e%perience #nder an optimal shipping sched#le. "he xij, 2i, and vi are not entered,
since they are #n)nown at the moment.
<e begin with x11 and assign it the minim#m of a1 = 1- and b1 = 3. "h#s, x11 = 3, leaving si% s#rpl#s cars at the first so#rce. <e
ne%t move one cell to the right and assign x12 = 2. "hese two allocations together e%ha#st the s#pply at the first so#rce, so we
move one cell down and consider x22. ?bserve, however, that the demand at the second destination has been satisfied by the x12
allocation. (ince we cannot deliver additional cars to it witho#t e%ceeding its demand, we m#st assign x22 = 4 and the move one
cell to the right. @ontin#ing in this manner, we obtain the degenerate sol#tion fewer than , 9 . M 1 = 2 positive entries! depicted
in "ablea# 1Q.
"o determine whether the initial allocation fo#nd in "ablea# 1Q is optimal, we first calc#late the terms #1 and vE with respect to
the basic&variable cells of the tablea#. Arbitrarily choosing #/ = 4 since the second row contains more basic variables that any
other row or col#mn, this choice will simplify the comp#tations!, we find:
/, /! cell: 22 + v2 = c22, 0 + v2 = 18, or v2 = 18
/, .! cell: 22 + v3 = c23, 0 + v3 = 1", or v3 = 1"
/, ,! cell: 22 + v4 = c24, 0 + v4 = 31, or v4 = 31
1, /! cell: 21 + v2 = c12, 21 + 18 = 17, or 21 = 51
1, 1! cell: 21 + v1 = c11, 51 + v1 = 45, or v1 = 46
., ,! cell: 23 + v4 = c34, 23 + 31 = 0, or 23 = 531
"hese val#es are shown in "ablea# 1@. De%t we calc#late the '#antities cij 2i vj for each nonbasic variable cell of "ablea# 1Q.
1, .! cell: c13 21 v3 = 21 =51> 1" = 3
1, ,! cell: c14 21 v4 = 30 =51> 31 = 0
/, 1! cell: c21 22 v1 = 14 0 46 = 532
., 1! cell: c31 23 v1 = 0 =531> 46 = 515
., /! cell: c32 23 v2 = 0 =531> 18 = 13
., .! cell: c33 23 v3 = 0 =531> 1" = 12
"hese res#lts also are recorded in "ablea# 1@, in parentheses.
..&-4
(ince at least one of these cij 2i vj!&val#es is negative, the c#rrent sol#tion is not optimal, and a better sol#tion can be obtained
by increasing the allocation to the variable cell! having the largest negative entry, here the /, 1! cell of "ablea# 1@. <e do so by
placing a boldface pl#s sign signaling an increase! in the /, 1! cell and identifying a loop containing, besides this cell, only
basic&variable cells. (#ch a loop is shown by the heavy lines in "ablea# 1@. <e now increase the allocation the /, 1! cell as m#ch
as possible, sim#ltaneo#sly adE#sting the other cell allocations in the loop so as not to violate the s#pply, demand, or nonnegativity
constraints. Any positive allocation to the /, 1! cell wo#ld force x22 to become negative. "o avoid this, b#t still ma)e x21 basic, we
assign x21 = 4 and remove x22 from o#r set of basic variables. "he new basic sol#tion, also degenerate, is given in "ablea# 1C.
<e now chec) whether this sol#tion is optimal. <or)ing directly on "ablea# 1C, we first calc#late the new 2i and vj with respect
to the new basic variables, and then comp#te cij 2i vj for each nonbasic variable cell. Again we arbitrarily choose 22 = 4, since
the second row contains more basic variables than any other row or col#mn. "hese res#lts are shown in parentheses in "ablea#
1$. (ince two entries are negative, the c#rrent sol#tion is not optimal, and a better sol#tion can be obtained by increasing the
allocation to the 1, ,! cell. "he loop whereby this is accomplished is indicted by heavy lines in "ablea# 1$= it consists of the cells
1, ,!, /, ,!, /, 1! and 1, 1!. Any amo#nt added to cell 1, ,! m#st be sim#ltaneo#sly s#btracted from cells 1, 1! and /, ,! and
then added to cell /, 1!, so as not to violate the s#pply&demand constraints. "herefore, no more than si% cars can be added to cell
1, ,! witho#t forcing %/, as a basic variable. "he new, nondegenerate basic sol#tion is shown in "ablea# 1I.
After
one
f#rther optimality test negative! and conse'#ent change of basis,
we obtain "ablea# 16, which also shown the res#lts of the
optimality test fo the new basic sol#tion. *t is seen that each cij 2i vj is nonnegative= hence the new sol#tion is optimal. "han is,
x12 = 6, x13 = 3, x14 = 6, x21 = ", x23 = 4, x34 = 3, with all other variables nonbasic and, therefore, 7ero. I#rthermore,
! = 211! 9 ./1! 9 2.4! 9 31,! 9 ,13! 9 .4! = 0-,1
"he fact that some positive allocation comes from the d#mmy so#rce indicates that not all demands can be met #nder this optimal
sched#le. *n partic#lar, destination , will receive three fewer cars than it needs.
.,&-4
Nonlinear Programming: Single5Variable 6ptimi7ation
!"e Problem
A one&variable, #nconstrained, nonlinear program has the form
optimi7e: ! = f=x> 1!
where f=x> is a nonlinear! f#nction of the single variable x, and the search for the optim#m ma%im#m or minim#m! is cond#cted
over the infinite interval &W, W!. *f the search is restricted to a finite s#binterval Ga, bH, then the problem becomes
optimi7e: ! = f=x>
s.t. a x b /!
which is one&variable, constrained program.
An obEective f#nction f=x> has a local or relative! minim#m at x0 if there e%ists a small! interval centered at x0 s#ch that f=x>
f=x0> for all x at which the f#nction is defined, then the minim#m at x0 besides being local! is a global or absol#te! minim#m.
Local and global ma%ima are defined similarly, in terms of the reversed ine'#ality.
Nrogram 1! see)s a global optim#m= program /! does too, insofar as it seems the best of the local optima over Ga, bH. *t is
possible that the obEective f#nction ass#mes even better val#es o#tside Ga, bH, b#t these are not of interest.
1esults $rom Calculus
"heorem 1: *f f=x> is contin#o#s on the closed and bo#nded interval Ga, bH, then f=x> has global optima both a ma%im#m and a
minim#m! on this interval.
"heorem /: *f f=x> has a local optim#m at x0 and if f=x> is differentiable on a small interval centered at x0, then fI=x0> = 0.
"heorem /: *f f=x> is twice&differentiable on a small interval centered at x0, and if fI=x0> = 0 and fII=x0> J 0, then f=x> has a local
minim#m at x0. *f instead fI=x0> = 0 and fII=x0> H 0, then f=x> has local ma%im#m at x0.
*f follows from the first two theorems that if f=x> is contin#o#s on Ga, bH, then local and global optima for program /! will occ#r
among points where fI=x0> does not e%ist, or among points where fI=x0> = 0 generally called stati-nar0 or critical points!, or
among the end points x = a and x = b.
(ince program 1! is not restricted to a closed and bo#nded interval, there are no endpoints to consider. *nstead, the val#es of the
obEective f#nction at the stationary points and at points where fI=x> does not e%ist are compared to the limiting val#es of f=x> as x
K L M. *t may be that neither limit e%ists consider f=x> = sin x!. Q#t if either limit does e%ist M and we accept L M as a XlimitY
here M and yields the best val#e of f=x> the largest for a ma%imi7ation program of the smallest for a minimi7ation program!, then a
global optim#m for f=x> does not e%ist. *f the best val#e occ#rs at one of the finite points, then this best val#e is the global
optim#m.
Se8uential5searc" !ec"ni8ues
*n practice, locating optima by calc#l#s is seldom fr#itf#l: either the obEective f#nction is not )nown analytically, so that
differentiation is impossible, or the stationary points cannot be obtained algebraically. *n s#ch cases, n#merical methods are #sed
to appro%imate the location of some! local optima to within an acceptable tolerance.
4eN2ential5searc, tec,niN2es start with a finite interval in which the obEective f#nction is pres#med 2ni.-1al= that is, the interval
is pres#med to incl#de one and only one point at which f=x> has a local ma%im#m or minim#m. "he techni'#es then systematically
shrin) the interval aro#nd the local optim#m #ntil the optim#m is confined to within acceptable limits= this shrin)ing is effected
by se'#entially eval#ating the obEective f#nction at selected points and then #sing the #nimodal property to eliminate portions of
the c#rrent interval.
(pecific se'#ential searches are considered in the three sections that follow.
!"ree5point Interval Searc"
"he interval #nder consideration is divided into '#arters and the obEective f#nction eval#ated at the three e'#ally spaced interior
points. "he interior point yielding the best val#e of the obEective is determined in case of a tie, arbitrarily choose one point!, and
the s#binterval centered at this point and made #p of two '#arters of the c#rrent interval replaces the c#rrent interval. "he three&
point interval search is the most efficient e'#ally spaced search proced#re in terms of achieving a prescribed tolerance with a
minim#m n#mber of f#nctional eval#ations. *t is also one of the easiest se'#ential searches to code for the comp#ter.
Fibonacci Searc"
"he Oib-nacci seN2ence, UOnV = U1, 1, /, ., -, A, 1., /1, .,, --, JV, forms the basis of the most efficient se'#ential&search
techni'#e. $ach n#mber in the se'#ence is obtained by adding together the two preceding n#mbers= e%ceptions are the first two
n#mber, O0 and O1, which are both 1.
"he Iibonacci search is initiali7ed by determining the smallest Iibonacci n#mber that satisfies OPB b a, where B is a prescribed
tolerance and Ga, bH is the original interval of interest. (et B F =b a> ) OP. "he first two points in the search are located OP 1BI
#nits in from the endpoint of the Ga, bH, where OP 1 is the Iibonacci n#mber preceding OP. (#ccessive points in the search are
considered one at a time and are positioned OjBIj = P 2, P 3, C, 2! #nits in from the ne+est endpoint of the c#rrent interval.
?bserve that with the Iibonacci proced#re we can state in advance the n#mber of f#nctional eval#ations that will be re'#ired to
achieve a certain acc#racy= moreover, that n#mber is independent of the partic#lar #nimodal f#nction.
Golden5ean Searc"
A search nearly as efficient as the Iibonacci search is based on the n#mber ( ) 21A4 . 4 / 1 - , )nown as the E-l1en
.ean. "he first two points of the search are located 4.21A4!b a! #nits in from the endpoints of the initial interval Ga, bH.
.-&-4
(#ccessive points are considered one at a time and are positioned 4.21A4Qi #nits in from the ne+est endpoint of the c#rrent
interval, where Qi denotes the length of this interval.
Convex Functions
(earch proced#res are g#aranteed to appro%imate global optima on a search interval only when the obEective f#nction is #nimodal
there. *n practice, one #s#ally does not )now whether a partic#lar obEective f#nction is #nimodal over a specified interval. <hen a
search proced#re is applied in s#ch a sit#ation, there is no ass#rance it will #ncover the desired global optim#m. $%ceptions
incl#de programs that have conve% or concave obEective f#nctions.
A f#nction f=x> is conve% on an interval R finite or infinite! if for any two points x1 and x2 in R and for all 0 S 1,
( ) ( ) ( ) ( ) ( )
/
1
1 /
1
1
x f x f x x f + +
.!
*f .! holds with the ine'#ality reversed, then f=x> is concave. "h#s, the negative of a conve% f#nction is concave, and conversely.
"he graph of a conve% f#nction is shown in Iig. ,= a defining geometrical property is that the c#rve lies on or above any of its
tangents. @onve% f#nctions and concave f#nctions are #nimodal.
"heorem ,: *f f=x> is twice&differentiable on R, then f=x> is conve% on R if and only if fII=x> 4 for all x in R. *t is concave if and
only if fII=x> 4 for all x in R.
"heorem -: *f f=x> is conve% on R, then any local minim#m on R is a global minim#m on R. *f f=x> is concave on R, then any local
ma%im#m on R is a global ma%im#m on R.
*f .! holds with strict ine'#ality e%cept at Z = 4 and Z = 1, the f#nctions is strictly conve%. (#ch a f#nction has a strictly positive
second derivative, and any local and therefore global! minim#m is #ni'#e. Analogo#s res#lts hold for strictly concave f#nctions.
Solved Problems
1. ;a%imi7e: ! = x=5T x> on G4, /4H
6ere f=x> = x=5T x> is contin#o#s, and fI=x> = 5T 2x. <ith the derivative defined everywhere, the global ma%im#m on G4, /4H
occ#rs at an endpoint x = 4 or x = /4, or at a stationary point, where fI=x> = 0. <e find x = 5T)2 as the only stationary point in G4,
/4H. $val#ating the obEective f#nction at each of these points, we obtain the table
A, . A- 23 . 21 4 !
/4 / - 4
x f
x
from which we concl#de that x = 5T)2, with ! = 61*6".
/. ;a%imi7e:
[ ] , , , on A
/
x !
.
6ere

'





x x
x x
x x
x x f
A A
A A A
A A
A !
/
/
/
/
is a contin#o#s f#nction, with

'

<
< <
<

x x
x x
x x
x f
A /
A A /
A /
!
"he derivative does not e%ist at A t x , and it is 7ero at x = 4= all three points are in GM,, ,H. $val#ating the obEective f#nction
at each of these points, and at the endpoints , t x , we generate the table
A 4 A 4 A !
, A 4 A ,
x f
x
from which we concl#de that the global ma%im#m on GM,, ,H is ! = A, which is ass#med at the three points , t x and x = 4.
.. :se the three&point interval search to appro%imate the location of the ma%im#m of f=x> = x=5T x> on G4, /4H to within [ = 1.
(ince fII=x> = 52 H 0 everywhere, it follows from "heorem ,! that f=x> is concave, hence #nimodal, on G4, /4H. "herefore, the
three&point interval search is g#aranteed to converge to the global ma%im#m.
First iteration: Cividing G4, /4H into '#arters, we have x1 = -, x2 = 14, and x3 = 1- as the three interior points. "herefore
f=x1> = x1=5T x1> = 5=5T 5> = 53*54
f=x2> = x2=5T x2> = 10=5T 10> = 57*08
f=x3> = x3=5T x3> = 15=5T 15> = 10*62
(ince x2 is the interior point generating the greatest val#e of the obEective f#nction, we ta)e the interval G-, 1-H, centered at x2 as
the new interval of interest.
Second iteration: <e divide G-, 1-H into '#arters, with x4 = 1.-, x2 = 14, x5 = 1/.- as the three interior points. (o
f=x4> = x4=5T x4> = 7*5=5T 7*5> = 61*56
f=x2> = x2=5T x2> = 10=5T 10> = 57*08 as before!
f=x5> = x5=5T x5> = 12*5=5T 12*5> = 40*10
.2&-4
As x4 is the interior point yielding the largest val#e of f=x>, we ta)e the interval G-, 14H, centered at x4, as the new interval of
interest.
Third iteration: <e divide G-, 14H into '#arters, with x6 = 2./-, x4 = 1.-, and x7 = A.1- as the new interior points. (o
f=x6> = x6=5T x6> = 6*25=5T 6*25> = 5"*11
f=x4> = 61*56 as before!
f=x7> = x7=5T x7> = 8*75=5T 8*75> = 60*88
As x4 yields the largest val#e of f=x>, we ta)e the interval G2./-, A.1-H. centered at x4, as the new interval of interest.
Fourth iteration: Cividing G2./-, A.1-H into '#arters, we generate x8 = 2.A1-, x4 = 1.-, and x" = A.1/- as the new interior points.
"h#s
f=x8> = x8=5T x8> = 6*875=5T 6*875> = 60*73
f=x4> = 61*56 as before!
f=x"> = x"=5T x"> = 8*125=5T 8*125> = 61*61
Dow x" is the interior point yielding the largest val#e of the obEective f#nction, so we ta)e the s#binterval centered at x", namely
G1.-, A.1-H, as the new interval for consideration. "he midpoint of this interval however, is within the prescribed tolerance, [ = 1,
of all other point in the interval= hence we ta)e
x = x" = A.1/- with ! = f=x"> = 21.21
,. Kedo Nroblem . #sing the Iibonacci search.
Initial points: "he first Iibonacci n#mber s#ch that OP=1> 20 0 is O8 = 21. <e set P = 8,
3-/, . 4
/1
4 /4


P
O
a b

and then position the first two points in the search


.A . 1/ ! 3-/, . 4 1.
1
O #nits
in from each endpoint. @onse'#ently,
x1 = 4 9 1/..A = 1/..A x2 = /4 M 1/..A = 1.2/
f=x1> = =12*38>=5T 12*38> = 41*20
f=x2> = =7*62>=5T 7*62> = 61*63
:sing the #nimodal property, we concl#de that the ma%im#m m#st occ#r to the left of 1/..A, we red#ce the interval of interest to
G4, 1/..AH.
First iteration: "he ne%t&lower Iibonacci n#mber O7 was the last one #sed! is O6 = A= so the ne%t point in the search is positioned
213 . 1 ! 3-/, . 4 A
2
O #nits
in from the newest endpoint, 1/..A. "h#s
x3 = 12*38 7*61" = 4*761
f=x3> = =4*761>=5T 4*761> = 52*12
<e concl#de that the ma%im#m m#st occ#r in the new interval of interest G,.121, 1/..AH.
Second iteration: "he ne%t&lower Iibonacci n#mber now is O5 = -. "h#s
! 3-/, . 4 -
-
O
x4 = 4*761 + 5=0*"524> = "*523
f=x4> = ="*523>=5T "*523> = 58*"0
<e concl#de that the new interval of interest is G,.121, 3.-/.H.
Third iteration: "he ne%t&lower Iibonacci n#mber now is O4 = .. 6ence
x5 = "*523 3=0*"524> = 6*666
f=x5> = =6*666>=5T 6*666> = 60*27
Irom the #nimodal property follows that the new interval of interest is G2.222, 3.-/.H
Fourth iteration: "he ne%t&lower Iibonacci n#mber now is O3 = /, 6ence
x6 = 6*666 + 2=0*"524> = 8*571
f=x6> = =8*571>=5T 8*571> = 61*17
<e concl#de that G2.222, A.-11H is the new interval of interest. "he midpoint of this interval, however, is within B = 1 in fact,
within B = 4.3-/,! of every other point of the interval. "heoretically, the midpoint sho#ld coincide with x2= the small apparent
discrepancy arises from ro#ndoff.! <e therefore accept x2 as the location of the ma%im#m, i.e.,
x = x2 = 7*62 with ! = f=x2> = 61*63
-. Kedo Nroblem . #sing the golden&mean search.
Initial points: "he length of the initial interval is Q1 = /4, so the first two points in the search are positioned
4.21A4!/4! = 1/..2 #nits
in from each endpoint. "h#s
x1 = 4 9 1/..2 = 1/..2 x2 = /4 M 1/..2 = 1.2,
f=x1> = =12*36>=5T 12*36> = 41*38
f=x2> = =7*64>=5T 7*64> = 61*64
Irom the #nimodal property follows that the ma%im#m m#st occ#r to the left of 1/..2= hence we retain G4, 1/..2H as the new
interval of interest.
First iteration: "he new interval has length Q2 = 1/..2, so the ne%t point in the search is positioned 4.21A4Q2 #nits in from the
newest end point. "herefore
.1&-4
x3 = 12*36 =0*6180>=12*36 = 4*722
f=x3> = =4*722>=5T 4*722> = 51*88
<e determine G,.1//, 1/..2H as the new interval of interest.
Second iteration: Q3 = 1/..2 M ,.1// = 1.2.A= th#s
x4 = 4*722 + =0*6180>=7*638> = "*442
f=x4> = ="*442>=5T "*442> = 5"*16
<e concl#de that G,.1//, 3.,,/H is the new interval of interest
Third iteration: Q4 = 3.,,/ M ,.1// = ,.1/4= th#s
x5 = "*442 =0*6180>=4*720> = 6*525
f=x5> = =6*525>=5T 6*525> = 5"*"2
<e concl#de that G2.-/-, 3.,,/H is the new interval of interest.
Fourth iteration: Q5 = 3.,,/ M 2.-/- = /.311= hence
x6 = 6*525 + =0*6180>=2*"17> = 8*328
f=x6> = =8*328>=5T 8*328> = 61*46
<e find G2.-/-, A../AH as the new interval of interest. Dotice that this new interval is of length less than /B = /, b#t that the
incl#ded sample point, x2, is not within B of all other points in the interval. "herefore, another iteration is re'#ired.
Fifth iteration: Q6 = A../A M 2.-/- = 1.A4.= therefore
x7 = 8*328 =0*6180>=1*803> = 7*214
f=x7> = =7*214>=5T 7*214> = 61*28
"his new point determines Gx7, x6H = G1./1,, A../AH as the new interval of interest. Dow, however, the interior point x2 = 1.2, is
within B = 1 of all other points in the interval= so we ta)e it as the location of the ma%im#m. "hat is,
x = x2 = 7*64 with ! = f=x2> = 61*64
Nonlinear Programming: ultivariable 6ptimi7ation %it"out Constraints
"he present chapter will largely consist in a generali7ation of the res#lts of single&variable optimi7ation to the case of more than
one variable.
optimi7e: ! = fO! where O F Gx1, x2, C, xnH
%
1!
;oreover, we shall always s#ppose the optimi7ation in 1! to be a ma%imi7ation= all res#lts will apply to a minimi7ation program
if fO! is replaced by M fO!.
)ocal and Global axima
Definition: An B&neighborhood B \ 4! aro#nd

]
is the set of all vectors O s#ch that
( ) ( ) ( ) ( ) ( )
/ / /
/ /
/
1 1
] ] ]
] ]
+ + +
n n
%
x x x x x x
*n geometrical terms, an B&neighborhood aro#nd

]
is the interior and bo#ndary of an n&dimensional sphere of radi#s B centered at

]
.
Gradient Vector and 9essian atrix
"he gradient vector
f
associated with a f#nction f=x1, x2, C, xn> having first partial derivatives is defined by
%
n
x
f
x
f
x
f
f
1
]
1

, , ,
/ 1

"he notation

]
f
signifies the val#e of the gradient at

]
. Ior small displacements from

]
in vario#s directions, the
direction of ma%im#m increase in fO! is the direction of the vector

]
f
.
Example & Ior ( )
.
.
/
/ /
/
1 . / 1
. , , x x x x x x x f , with [ ]
%
. , / , 1
]
,
1
1
1
]
1


/
.
/
/
.
. /
/
1
/ 1
.
/ .
2
x x
x x x
x x
f whence
1
1
1
]
1


1
1
1
]
1


14A
14-
1/
! . ! / .
! . ! / / ! 1 .
! / ! 1 2
/ /
. /
f
"herefore, at G1, /, .H
"
, the f#nction increases most rapidly in the direction of G1/, M14-, M14AH
"
.
"he Uessian .atrix associated with a f#nction f=x1, x2, C, xn> that has second partial derivatives is
1
1
]
1


j i
f
x x
f
/
i, j = 1, 2,C,n!
"he notation

]
f signifies the val#e of the 6essian matri% at

]
. *n preparation for "heorems , and - below, we shall need
the following:
Definition: An n T n symmetric matri% A one s#ch that A = A
"
! is negative definite negative semi&definite! if O
"
AO is negative
nonpositive! for every n&dimensional vector O ^ 4.
!"eorem &: Let A F GaijH be an n T n symmetric matri%, and define the determinants
.A&-4
( ) +

det 1
1
.. /. .1
./ // /1
.1 /1 11
.
// /1
1/ 11
/ 11 1
n
n
'
a a a
a a a
a a a
'
a a
a a
' a '
"hen A is negative definite if and only if '1, '2, C, 'n are all negative= A is negative semi&definite if and only if '1, '2,
C, 'r =r H n> are all negative and the remaining 'Bs are all 7ero.
Example ' Ior the f#nction of $%ample 1,
1
1
1
]
1



.
/
/
/
. /
/
. /
.
. 1
1 /
2 2 4
2 / 2
4 2 2
x x x x
x x x x
x x
f
whence
1
1
1
]
1

1/ 14A 4
14A -, 2
4 2 1/
]
f
Ior

]
f , '1 = 1/ \ 4, so that f

is not negative definite, or even negative semi&definite, at

]
.
1esults $rom Calculus
!"eorem ': *f fO! is contin#o#s on a closed and bo#nded region, then fO! has a global ma%im#m and also a global minim#m!
on that region.
!"eorem .: *f fO! has a local ma%im#m or a local minim#m! at O
L
and if
f
e%ists on some B&neighborhood aro#nd O
L
, then
. 4

f
!"eorem /: *f fO! has second partial derivatives on an B&neighborhood aro#nd O
L
, and if
4

f
and

f is negative
definite, then fO! has a local ma%im#m at O
L
.
*t follows from "heorem / and . that a contin#o#s fO! ass#mes its global ma%im#m among those points at which
f
does not
e%ist or among those points at which
4 f
stationary points! M #nless the f#nction ass#mes even larger val#es as O
"
O _ W.
*n the latter case, no global ma%im#m e%ists.
Analytical sol#tions based on calc#l#s are even harder to obtain for m#ltivariable programs than for single&variable programs, and
so, once again, n#merical methods are #sed to appro%imate local! ma%ima to within prescribed tolerance.
!"e et"od o$ Steepest #scent
@hoose an initial vector O4, ma)ing #se of any prior information abo#t where the desired global ma%im#m might be fo#nd. "hen
determine vectors O1, O/, O., J by the iterative relation
k
f
k k k

+
+
1 /!
6ere

k
is a positive scalar which ma%imi7es
( )
k
f f
k

+
= this single&variable program is solved by the methods of
single&variable optimi7ation. *t is best if

k
represents a global ma%im#m= however, a local ma%im#m will do. "he iterative
process terminates if and when the differences between the val#es of the obEective f#nction at two s#ccessive O&vectors is smaller
than a prescribed tolerance. "he last&comp#ted O&vector becomes the final appro%imation to OL.
!"e Ne%ton51ap"son et"od
@hoose an initial vector O4, as in the method of steepest ascent. +ectors O1, O/, O., J then are determined iteratively by
k
k
f
f k k

,
_


1
1
.!
"he stopping r#le is the same as for the method of steepest ascent.
"he Dewton&Kaphson method will converge to a local ma%im#m if f

is negative definite on some B&neighborhood aro#nd the


ma%im#m and if O4, lies in that B&neighborhood.
1emark &: *f f

is negative definite,
1

f
e%ists and is negative definite.
*f O4 is not chosen correctly, the method may converge to a local minim#m or it may not converge at all. *n either case, the
iterative process is terminated and then beg#n anew with a better initial appro%imation.
!"e Fletc"er5Po%ell et"od
"his method, an eight&step algorithm, is beg#n by choosing an initial vector

]
and prescribing a tolerance B, and by setting an n
T n matri% > e'#al to the identity matri%. Qoth

]
and > are contin#ally #pdated #ntil s#ccessive val#es of the obEective f#nction
differ by less than B, where#pon the last val#e of

]
is ta)en as O
L
.
(tep1: $val#ate !
]
f and
.
]

f (
(tep/: Cetermine

s#ch that ( ) + >
]
f is ma%imi7ed when .

(et . > C

(tep.: Cesignate
C
]
+
as the #pdated val#e of .
]

(tep,: @alc#late !
]
f for the #pdated val#es of .
]
*f
, <
go to (tep-= if not, go to (tep2.
(tep-: (et , ! ,
]


f and stop.
(tep2: $val#ate @ =

]
f
for the #pdated vector ,
]
and set ` = Q M @.
.3&-4
(tep1: @alc#late the n T n matrices
> >``
>` `
1 &
; and CC
` C
1
L
"
"

,
_


,
_

%
%
(tepA: Cesignate > 9 L 9 ; as the #pdated val#e of >. (et

e'#al to the c#rrent val#e of


,
Q e'#al to the c#rrent val#e of @,
and ret#rn to (tep/.
9ooke5:eeves3 Pattern Searc"
"his method is a direct&search algorithm that #tili7es ex3l-rat-r0 .-ves, which determine an appropriate direction, and 3attern
.-ves, which accelerate the search. "he method is beg#n by choosing an initial vector, Q F Gb1, b2, C, bnH
"
, and step si7e, ,.
(tep1: $%ploratory moves aro#nd Q are made by pert#rbing the components of Q, in se'#ence, by a, #nits. *f either pert#rbation
improves bi.e., increases! the val#e of the obEective f#nction beyond the c#rrent val#e, the initial val#e being fQ!, the
pert#rbed val#e of that component is retained= otherwise the original val#e of the component is )ept. After each component
has been tested in t#rn, the res#lting vector is denoted by @. *f @ = Q, go to (tep/= otherwise go to (tep..
(tep/: Q is the location of the ma%im#m to within a tolerance of ,. $ither , is red#ced and (tep1 repeated, or the search is
terminated with O
L
= Q.
(tep.: ;a)e a pattern move to a temporary vector " = /@ M Q. " is reached by moving from Q to @ and contin#ing for an e'#al
distance in the same direction.!
(tep,: ;a)e e%ploratory moves aro#nd " similar to the ones aro#nd Q described in (tep1. @all the res#lting vector (. *f ( = ", go
to (tep2.
(tep-: (et Q = @ and ret#rn to (tep1.
(tep2= (et Q = @, @ = (, and ret#rn to (tep..
# odi$ied Pattern Searc"
6oo)e&ceevesB pattern search terminates when no pert#rbation of any one component of Q leads to an improvement in the
obEective f#nction. ?ccasionally this termination is premat#re, in that pert#rbations of two or more of the components
sim#ltaneo#sly may lead to an improvement in the obEective f#nction. (im#ltaneo#s pert#rbations can be incl#ded in the method
by modifying (tep/ as follows:
(tep/I: @ond#ct an e%ha#stive search over the s#rface of the hyperc#be centered at Q by k, #nits, where k = M1, 4, 1. Ior a vector
of n components, there are .
n
M 1 pert#rbations to consider. As soon as an improvement is reali7ed, terminate the e%ha#stive
search, set the improved vector e'#al to Q, and ret#rn to (tep1. *f no improvement is reali7ed, Q is the location of the
ma%im#m to within a tolerance of ,. $ither , is red#ced and (tep1 repeated, or the search is terminated with O
L
= Q.
C"oice o$ an Initial #pproximation
$ach n#merical method starts with a first appro%imation to the desired global ma%im#m. At times, s#ch an appro%imation is
apparent from physical or geometrical aspects of the problem. *n other cases, a random n#mber generator is #sed to provide
different val#es for O. "hen fO! is calc#lated for each randomly chosen O, and that O which yields the best val#e of the obEective
f#nction is ta)en as the initial appro%imation. $ven this random sampling proced#re implies and initial g#ess of the location of the
ma%im#m, in that the random n#mbers m#st be normali7ed so as to lie in some fi%ed interval.
Concave Functions
"here is no g#arantee that a n#merical method will #ncover a global ma%im#m. *t may converge to merely a local ma%im#m or,
worse yet, may not converge at all. $%ceptions incl#de programs having concave obEective f#nctions.
A f#nction fO! is conve% on a conve% region R, if for any two vector O1 and O/ in R and for all 4 S 1,
fSO1 9 1 M S!O/! SfO1! 9 1 M S!fO/! ,!
A f#nction is concave on R if and only if its negative is conve% on R. "he conve% region R may be finite or infinite.
!"eorem 0: *f fO! has second partial derivatives on R, then fO! is concave on R if and only if its 6essian matri% f

is
negative semi&definite for all O in R.
!"eorem ;: *f fO! is concave on R, then any local ma%im#m on R is a global ma%im#m on R.
"hese two theorems imply that, if f

is negative semi&definite everywhere, then any local ma%im#m yields a sol#tion to


program 1!. *f f

is negative definite everywhere, then fO! is strictly concave everywhere!, and the sol#tion to program 1! is
#ni'#e.
Solved Problems
1. ;a%imi7e: ! . ! 1
/
. . / 1
+ x x x x !
6ere !. . ! 1 ! , ,
/
. . / 1 . / 1
+ x x x x x x x f "he gradient vector,
[ ] , . . , , 1
/
. 1 /
%
x x x f
e%ists everywhere
and is 7ero only at
O1 = G4, 1, 1H
%
and O/ = G4, 1, M 1H
%
<e have fO1! = M / and fO/! = /. Q#t f=x1, x/, x.! becomes arbitrarily large as x. for instance! does so= hence no global ma%im#m
e%ists. "he vector O/ is not even the site of a local ma%im#m= rather, it is a saddle point, as is O1.
/. ;inimi7e:
( ) ( ) . 14 -
/
/
/
1
+ + x x !
;#ltiplying this obEective f#nction by M 1, we obtain the e'#ivalent ma%imi7ation program
,4&-4
;a%imi7e:
( ) ( ) 14 -
/
/
/
1
x x !
for which
[ ] . , - /
/ 1
%
x x !
"h#s there is a single stationary point, , , -
/ 1
x x at which ! = M 14. Dow, as
,
/
/
/
1
+ x x ! becomes arbitrarily small= conse'#ently, !
L
= M 14 is the global ma%im#m, and !
L
= 9 14 is the global minim#m
for the original minimi7ation program. "he minim#m is, of co#rse, also ass#med as . 1,12 . . , /.21 . / -
/ 1


x x
.. :se the method of steepest ascent to
minimi7e:
( ) ( ) 14 -
/
/
/
1
+ + x x !
>oing over to the e'#ivalent program
ma%imi7e:
( ) ( ) 14 -
/
/
/
1
x x !
1!
we re'#ire a starting program sol#tion, which we obtain by a random sampling of the obEective f#nction over the region M 14 x1,
x2 14. "he sample points and corresponding !&val#es are shown in the table below. "he ma%im#m !&entry is M .2..A, occ#rring
at O4 = G2.-21, -.A31H
%
, which we ta)e as the initial appro%imation to O
L
. "he gradient of the obEective f#nction for program 1! is
( )
( )
1
1
]
1

/
1
/
- /
x
x
f
x1 M A.-.1 M 4.313A 3./41 3./-4 2.-31 A.,11 A./4/ M 3.11. M 3...1 M -.13,
x2 M 1.433 M A.44- M /.-/, 1.-,2 -.A31 M 3.3,- M -.143 M 2.31, A.12. M 4.4/14
! M 1,,.4 M 1,,./ M 34.21 M 1A.-3 M .2.-A M /13., M 1/..3 M /,1.. M 123./ M A,.,A
First iteration.
( )
( )
1
]
1

1
1
]
1



+
1
]
1


,33 . - A31 . -
1// . A -31 . 2
A31 . - /
- -31 . 2 /
A31 . -
-31 . 2
4
4
f
( ) ( ) ( )
-3 . .2 . . 142 . . 142
14 ,33 . - A31 . - - 1// . A -31 . 2
/
/
/
4
4
+
+


f f
:sing the analytical methods described in previo#s chapter, we determine that this f#nction of ass#mes a global! ma%im#m at
. - . 4
4

"h#s,
( )
( )
1
]
1

1
]
1

1,/ . .
/.2 . /
- . 4 ,33 . - A31 . -
- . 4 1// . A -31 . 2
4
4 4 1
f
with fO1! = M 14.44. (ince the difference between fO4! = M .2.-A and fO1! = M 14.44 is significant, we contin#e iterating.
Second iteration.
( )
( )
1
]
1

1
1
]
1



+
1
]
1


444A . 4 1,/ . .
4441 . 4 /.2 . /
1,/ . . /
- /.2 . / /
1,/ . .
/.2 . /
1
1
f
( ) ( ) ( )
( )
1 A /
/
/
1
14 14 .A/ . 2 -44 . 2
14 444A . 4 1,/ . . - 4441 . 4 /.2 . /
1

+
+

f f
:sing the analytical methods described in previo#s chapter, we find that this f#nction of has a global! ma%im#m at
. ,343 . 4
1

"h#s,
( )
( )
1
]
1

1
]
1

+
+

1,/ . .
/.2 . /
,343 . 4 444A . 4 1,/ . .
,343 . 4 4441 . 4 /.2 . /
1
1 1 /
f
(ince O1 = O/ to fo#r significant fig#res!, we accept O
L
= G/./.2, ..1,/H
%
, with !
L
= M 14.44, as the sol#tion to program 1!. "he
sol#tion to the original minimi7ation program is then O
L
= G/./.2, ..1,/H
%
, with !
L
= 9 14.44. @ompare this with the res#lts in
Nroblem /.
,. :se the Dewton&Kaphson method to
ma%imi7e:
( ) ( ) 14 -
/
/
/
1
+ + x x !
to within a tolerance of 4.4-
Irom Nroblem . we ta)e the initial appro%imation O4 = G2.-21, -.A31H
%
, with fO4! = M .2.-A. "he gradient vector, 6essian matri%,
and inverse 6essian matri% for this obEective f#nction are, respectively,
,1&-4
( )
( )
1
1
]
1

/
1
/
- /
x
x
f
1
]
1


/ 4
4 /
f 1
]
1

- . 4 4
4 - . 4
1
f
for all x1 and x2.
First iteration.
( )
( )
1
]
1

1
1
]
1

,33 . -
1// . A
A31 . - /
- -31 . 2 /
4

f
4
4
1
4 1

,
_

f
f
1
]
1

1
]
1

1
]
1

1
]
1

1,/ . .
/.2 . /
,33 . -
1// . A
- . 4 4
4 - . 4
A31 . -
-31 . 2
with fO1! = M 14.44. (ince
fO1! M fO4! = M 14.44 M M .2.-A! = /2.-A \ 4.4-
we contin#e iterating.
Second iteration.
( )
( )
1
]
1

1
1
]
1

444A . 4
4441 . 4
1,/ . . /
- /.2 . / /
1

f
1
1
1
1 /

,
_

f
f
1
]
1

1
]
1

1
]
1

1
]
1

1,/ . .
/.2 . /
444A . 4
4441 . 4
- . 4 4
4 - . 4
1,/ . .
/.2 . /
with fO/! = M 14.44. (ince fO/! M fO1! = 4 P 4.4-, we ta)e O
L
= O/ = G/./.2, ..1,/H
%
, with !
L
= fO/! = M 14.44.
-. A maEor oil company wants to b#ild a refinery that will be s#pplied from three port cities. Nort Q is located .44 )m east and ,44
)m north of Nort A, while Nort @ is ,44 )m east and 144 )m so#th of Nort Q. Cetermine the location of the refinery so that the
total amo#nt of pipe re'#ired to connect the refinery to the ports is minimi7ed.
"he obEective is tantamo#nt to minimi7ing the s#m of the distances between the refinery and the three ports. As an aid to
calc#lating this s#m, we establish a coordinate system, with Nort A as the origin. *n this system, Nort Q has coordinates .44, ,44!
and Nort @ has coordinates 144, .44!. <ith x1, x2! designating the #n)nown coordinates of the refinery, the obEective is
minimi7e:
( ) ( ) ( ) ( )
/
/
/
1
/
/
/
1
/
/
/
1
.44 144 ,44 .44 + + + + + x x x x x x !
(olve the problem to within 4./- )m by the Iletcher&Nowell method.
"his problem is e'#ivalent to a ma%imi7ation program with obEective f#nction
( ) ( ) ( ) ( ) ( )
/
/
/
1
/
/
/
1
/
/
/
1
.44 144 ,44 .44 + + + x x x x x x f
1!
and gradient vector
( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) 1
1
1
1
1
]
1


/
/
/
1
/
/
/
/
1
/
/
/
/
1
/
/
/
/
1
1
/
/
/
1
1
/
/
/
1
1
.44 144
.44
,44 .44
,44
.44 144
144
,44 .44
.44
x x
x
x x
x
x x
x
x x
x
x x
x
x x
x
f /!
"o initiali7e the Iletcher&Nowell method, we set B = 4./- and
1
]
1

1 4
4 1
>
and choose [ ] , /44 , ,44
]
%
which appears to be a good appro%imation to the optimal location of the refinery.
4te31:
( ) ( ) /44 , ,44
]
f f
( ) ( ) ( ) ( ) ( ) ( ) 4- . 3A1 144 .44 /44 144 /44 ,44
/ / / / / /
+ + +
1
]
1

12.,, . 4
.3/32 . 4
]
f
4te32:
( )

,
_

1
]
1

,
_

1
]
1

1
]
1

+
1
]
1


12.,, . 4 /44
.3/32 . 4 ,44
12.,, . 4
.3/32 . 4
1 4
4 1
/44
,44
>
]
f f f
,/&-4
( ) ( )
( ) ( )
( ) ( )
/ /
/ /
/ /
12.,, . 4 144 .3/32 . 4 .44
12.,, . 4 /44 .3/32 . 4 144
12.,, . 4 /44 .3/32 . 4 ,44



+ +
+ +
+ +
;a)ing a three&point interval search of G4, ,/-H, we determine V
L
d /1/.-. "herefore,
( )
1
]
1

1
]
1

1
]
1



/. . 12/
-4, . A.
132,, . 4
.3/32 . 4
1 4
4 1
- . /1/ > C
4te33:
1
]
1

1
]
1

+
1
]
1

+
/. . .2/
-4 . .12
/. . 12/
-4, . A.
/44
,44
C
]
which we ta)e as the #pdated [ ] . /. . .2/ , -4 . .12
]
:
]
%

4te34:
( ) ( ) 12 . 314 /. . .2/ , -4 . .12
]
f f
( ) /- . 4 /3 . 12 4- . 3A1 12 . 314 >
4te36:
1
]
1

44.1-3, . 4
411/41 . 4
@
]
f
1
]
1

1
]
1

1
]
1


124/A . 4
./11- . 4
44.1-3, . 4
4411/41 . 4
1.2,, . 4
.3/32 . 4
@
4te37:
[ ]
[ ]
[ ]
[ ]
1
]
1

1
]
1

1
]
1

1
]
1

1
]
1

1
]
1

1
]
1


1
]
1

1
]
1

1
]
1

1
]
1


A,A11 . 4 .-A3/ . 4
.-A3/ . 4 1-1A3 . 4
-1A4. . 4 /,,2/ . 4
/,,2/ . 4 14.-/ . 4
2A1-- . 4
1

1 4
4 1
124/A . 4 , ./11- . 4
124/A . 4
./11- . 4
1 4
4 1
2A1-- . 4
1

> >
2A1-- . 4
1
2A1-- . 4
124/A . 4
./11- . 4
1 4
4 1
124/A . 4 , ./11- . 4 >
/1 . 11- 1A1 . 34
1A1 . 34 ,/1 . ,2
/2.13 1.-,1
1.-,1 3 . 231/
/1 . 1-4
1

/. . 12/ , -4, . A.
/. . 12/
-4, . A.
/1 . 1-4
1
CC
/1 . 1-4
1
L
/1 . 1-4
124/A . 4
./11- . 4
/. . 12/ , -4, . A. C
%
%
%
%
4te38:
1
]
1

1
]
1

+
1
]
1

+
1
]
1

+ +
.2 . 11- A/A . A3
A/A . A3 /23 . ,1
A,A11 . 4 .-A3/ . 4
.-A3/ . 4 1-1A3 . 4
/1 . 11- 1A1 . 34
1A1 . 34 ,/1 . ,2
1 4
4 1
L >
which we ta)e as the #pdated >. <e also #pdate S = M 314.12 and
1
]
1


44.1-3, . 4
411/41 . 4
4te32:
( )
( ) ( )
( ) ( )
( ) ( )
/ /
/ /
/ /
3-4, . 2 /. . 2/ 2,31 . . -4 . .A.
3-4, . 2 11 . .1 2,31 . . -4 . 12
3-4, . 2 /. . .2/ 2,31 . . -4 . .12
3-4, . 2 /. . .2/
2,31 . . -4 . .12

44.1-3, . 4
411/41 . 4
.2 . 11- A/A . A3
A/A . A3 /23 . ,1
/. . .2/
-4 . .12
>
]



+ +
+ +
+ +

,
_

1
]
1

,
_

1
]
1

1
]
1

+
1
]
1

+
f
f f
;a)ing a three&point interval search of G4, 14H, we determine V
L
d 1./-. "herefore,
,.&-4
( )
1
]
1

1
]
1

1
]
1



2AA4 . A
-2/1 . ,
44.1-3, . 4
411/41 . 4
.2 . 11- A/A . A3
A/A . A3 /23 . ,1
/- . 1 > C
4te33:
1
]
1

1
]
1

+
1
]
1

+
3/ . .14
3, . .11
2AA4 . A
-2/1 . ,
/. . .2/
-4 . .12
C
]
which we ta)e as the #pdated .
]

4te34:
( ) ( )
( ) /- . 4 1A . 4 12 . 31 -A . 314
-A . 314 3/ . .14 , 3, . .11
]
<


f f
4te35:
1
]
1

3/ . .14
31 . .11
]
and ( ) -A . 314

f
"h#s the problem is solved by )m. -A . 314 with )m, 3/ . .14 )m, 31 . .11
/ 1
+

! x x
Nonlinear Programming: ultivariable 6ptimi7ation %it" Constraints
Standard Forms
<ith O F Gx1, x2, C, xnH
%
, standard form for nonlinear programs containing only e'#ality constraints is
ma%imi7e: ! = fO!
s.t. E1O! = 4
E2O! = 4 1!
JJJ..
E.O! = 4
with: . P n fewer constraints than variables!
As in previo#s chapter minimi7ation programs are converted into ma%imi7ation programs by m#ltiplying the obEective f#nction by
M 1.
(tandard form for nonlinear programs containing only ine'#ality constraints is either
ma%imi7e: ! = fO!
s.t. E1O! 4
E2O! 4 /!
JJJ..
E.O! 4
or
ma%imi7e: ! = fO!
E1O! 4
E2O! 4 .!
JJJ..
E.O! 4
with: O 4
"he two forms are e'#ivalent: /! goes over into .! with . = 3! #nder the s#bstit#tion O = : M +, with : 4 and + 4= on the
other hand, .! is E#st /! in the special case 3 = . + n and E. 9 iO! = M xi i = 1, /, J, n!. Iorm .! is appropriate when the
sol#tion proced#re re'#ires nonnegative variables. *n 1!, /!, or .!, f is a nonlinear f#nction, b#t some or all of the EBs may be
linear.
Donlinear programs not in standard form are solved either by p#tting them in s#ch form or by s#itably modifying the sol#tion
proced#re given below for programs in standard form.
)agrange ultipliers
"o solve program 1!, first form the QaEranEian f2ncti-n
( ) ( ) ( )

.
i
i i . n
E f x x x Q
1
/ 1 / 1
, , , , , , , ,!
where Vi i = 1, /, J, .! are #n)nown! constants called QaEranEe .2lti3liers. "hen solve the system of n + . e'#ations
( )
( ) . i
Q
n j
x
Q
i
j
, , / , 1 4
, , / , 1 4

-!
"heorem 1: *f a sol#tion to program 1! e%ists, it is contained among the sol#tions to system -!, provided fO! and EiO! i = 1, /,
J, .! all have contin#o#s first partial derivatives and the . T n Wac-bian .atrix,
1
1
]
1

j
i
x
E
c
has ran) . at O = O
L
.
,,&-4
"he method of Lagrange m#ltipliers is e'#ivalent to #sing the constraint e'#ations to eliminate certain of the x&variables from the
obEective f#nction and then solving an #nconstrained ma%imi7ation problem in the remaining x&variables.
Ne%ton51ap"son et"od
(ince ( ) ( ) Q x x x Q
. n
, , , , , , ,
/ 1 / 1
is nonlinear, it is #s#ally impossible to solve -! analytically. 6owever, since
the sol#tions to -! are the stationary points of Qe!, and since "heorem . of previo#s chapter! the ma%ima and minima of Qe!
occ#r among these stationary points, it sho#ld be possible to #se the Dewton&Kaphson method see previo#s chapter! to
appro%imate the XrightY e%trem#m of Qe!= that is, the one that corresponds to the optimal sol#tion of 1!. "he iterative form#la
applicable here is
k
k
f
f k k

,
_


1
1
2!
"his approach is of limited val#e beca#se, as in previo#s chapter, it is very diffic#lt to determine an ade'#ate e4. Ior an incorrect
e4, the Dewton&Kaphson method may diverge or may converge to the XwrongY e%trem#m of Qe!. *t is also possible for the
method to converge when no optimal sol#tion e%ists.
Penalty Functions
An alternative approach to solving program 1! involves the #nconstrained program
ma%imi7e: ( ) ( )

.
i
i i
E 3 f !
1
/
]
1!
where 3i \ 4 are constants still to be chosen! called 3enalt0 +eiE,ts. "he sol#tion to program 1! in the sol#tion to program 1!
when each EiO! = 4. Ior large val#es of the 3i, the sol#tion to 1! will have each EiO! near 7ero to avoid adverse effects on the
obEective f#nction from the terms ( )=
/

i i
E 3 and as each ,
i
3 each ( ) . 4
i
E
*n practice, this process cannot be accomplished analytically e%cept in rare cases. *nstead, program 1! is solved repeatedly by the
modified pattern search described in previo#s chapter, each time with either a new set of increased penalty weights or a decreased
step si7e. $ach pattern search with a specified set of penalty weights and a given step si7e is one phase of the sol#tion proced#re.
"he starting vector for a partic#lar phase is the final vector from the phase immediately preceding it. Nenalty weights for the first
phase are chosen small, often 15-4 = 4.4/= the first step si7e generally is ta)en as 1.
@onvergence of this proced#re is affected by the rates at which the penalty weights are increased and the step si7e is decreased.
Cecisions governing these rates are more a matter of art than of science.
2u"n5!ucker Conditions
"o solve program .!, first rewrite the nonnegativity conditions as M x1 4, M x2 4, J, M xn 4, so that the constraint set is . + n
ine'#ality re'#irements each with a less than or e'#als sign. De%t add slac) variables , , , ,
/
/
/
/
/
1 . n n n
x x x
+ + +

respectively, to the left&hand sides of the constraints, thereby converting each ine'#ality into an e'#ality. 6ere the slac) variables
are added as s'#ared terms to g#arantee their nonegativity. "hen form the Lagrangian f#nction
( ) ( ) [ ] [ ]
+ +

+
+
+ +
.
i
n .
. i
i n i i i n i i
x x x E f Q
1 1
/ /
A!
where
n .+
, , ,
/ 1
are Lagrange m#ltipliers. Iinally solve the system
( ) . n j
x
Q
j
+

/ , /, 1, 4
3!
( ) . n i
Q
i
+

/ , /, 1, 4

14!
( ) . n j
i
+ / , /, 1, 4 11!
$'#ations 3! thro#gh 11! constit#te the D2,n5%2cker c-n1iti-ns for program /! and .!."he first two sets, 3! and 14!, follow
directly from Lagrange m#ltiplier theory= set 11! is )nown as the constraint '#alification. Among the sol#tions to the S#hn&
"#c)er conditions will be the sol#tion to program .! if fO! and each EiO! have contin#o#s first partial derivatives.
et"od o$ Feasible (irections
"his is a five&step algorithm for solving program /!. "he method is applicable only when the feasible region has an interior, and
then it will converge to the global ma%im#m only if the initial appro%imation is XnearY the sol#tion. "he feasible region will have
no interior if two of the ine'#ality constraints have arisen from the conversion of an e'#ality constraint.
(tep1: Cetermine an initial, feasible appro%imation to the sol#tion, designating it Q.
(tep/: (olve the following linear program for the 11, 12, C, 1n 9 1:
ma%imi7e: ! = 1n + 1
s.t.
,-&-4
( )
( )
( ) +

+ +

+ +

+ +

+

+

+

+

3 n 3 n
n
3 3 3
n n
n
n n
n
n
n
E 1 k 1
x
E
1
x
E
1
x
E
E 1 k 1
x
E
1
x
E
1
x
E
E 1 k 1
x
E
1
x
E
1
x
E
1
x
f
1
x
f
1
x
f
1 /
/
1
1
/ 1 /
/
/
/
/
1
1
/
1 1 1
1
/
/
1
1
1
1
1 /
/
1
1
4

1/!
with: 1j 1 j = 1, /, J, n 9 1!
6ere ki i = 1, /, J, 3! is 4 if EiO! is linear and 1 if EiO! is nonlinear.
(tep.: *f 1n + 1 = 4, then O
L
= Q= if not, go to (tep ,.
(tep,: (et C = G11, 12, C, 1nH
%
. Cetermine a nonnegative val#e for V that ma%imi7es fQ 9 VC! while )eeping Q 9 VC feasible=
designate this val#e as V
L
.
(tep-: (et Q = Q 9 V
L
C and ret#rn to (tep/.
(olved Nroblems
1. ma%imi7e: ! = /x1 +x1 x/ + .x.
s.t. x1
/
9 x/ = .
*t is apparent that for any large negative x1 there is a large negative x/ s#ch that the constraint e'#ation is satisfied. Q#t then
! d x1x/ _ W. "here is no global ma%im#m.
/. minimi7e: ! = x1 9 x/ 9 x.
s.t. x1
/
9 x/ = .
x1 9 .x/ 9 /x. = 1
"he given program is e'#ivalent to the #nconstrained minimi7ation of
7 = f x1
/
9 x1 9 ,!
which obvio#sly has a sol#tion. <e may therefore apply the method of Lagrange m#ltipliers to the original program standardi7ed
as
ma%imi7e ! = M x1 M x/ M x.
s.t. x1
/
9 x/ M . = 4 1!
x1 9 .x/ 9 /x. M 1 = 4
6ere, fx1, x/, x.! = M x1 M x/ M x., n = . variables!, . = / constraints!,
E1x1, x/, x.! = x1
/
9 x/ M . E/x1, x/, x.! = x1 9 .x/ 9 /x. M 1
"he Lagrangian f#nction is then
Q = M x1 M x/ M x.! M V1x1
/
9 x/ M .! M V/x1 9 .x/ 9 /x. M 1!
and system -! becomes
( )
( )
( )
( ) ( )
( ) ( ) 2 4 1 / .
- 4 .
, 4 / 1
. 4 . 1
/ 4 / 1
. / 1
/
/
/
1
1
/
.
/ 1
/
/ 1 1
1
+ +

x x x
Q
x x
Q
x
Q
x
Q
x
x
Q



(#ccessively solving ,! for V/, .! for V1, /! for x1, -! for x/, and 2! for x., we obtain the #ni'#e sol#tion V/ = M 4.-, V1 = 4.-, x1 =
M 4.-, x/ = /.1-, and x. = M 4..1-, with
! = M x1 M x/ M x. = M M 4.-! M /.1- M M 4..1-! = M 1.A1-
(ince the first partial derivatives of fx1, x/, x.!, E1x1, x/, x.!, and E/x1, x/, x.! are all contin#o#s, and since
1
]
1

1
1
1
1
]
1

/ . 1
4 1 /
c
1
.
/
/
/
1
/
.
1
/
1
1
1
x
x
E
x
E
x
E
x
E
x
E
x
E
is of ran) / everywhere the last two col#mns are linearly independent everywhere!, either x1 = M 4.-, x/ = /.1-, x. = M 4..1- is the
optimal sol#tion to program 1! or no optimal sol#tion e%ists. @hec)ing feasible points in the region aro#nd M 4.-, /.1-, M 4..1-!,
,2&-4
we find that this point is indeed the location of a global! ma%im#m for program 1!. "herefore, it is also the location of a global
minim#m for the original program, with !
L
= M M 1.A1-! = 1.A1-.
.. Cisregarding Nroblem 1 #se the Dewton&Kaphson method to
ma%imi7e: ! = /x1 +x1 x/ + .x/
s.t. x1
/
9 x/ = .
6ere, Q = /x1 9 x1x/ 9 .x/! M V1x1
/
9 x/ M .!. "herefore,
1
1
1
]
1

+
+
+

.
.
/ /
/
/
1
1 1
1 1 /
x x
x
x x
Q

1
1
1
]
1



4 1 /
1 4 1
/ 1 /
1
1 1
x
x
Q

Arbitrarily ta)ing e4 = G1, 1, 1H


%
, we calc#late:
Oirst iterati-n
1
1
1
]
1


1
.
1
4
X
Q
1
1
1
]
1



4 1 /
1 4 1
/ 1 /
4
X
Q
( )
1
1
1
]
1






1 , 1
, , /
1 / 1
2
1 1
4
X
Q
and
( ) [ ]
%
Q
Q . 14 , . 14 , . 1
4 4
1
4 1

4ec-n1 iterati-n
1
1
1
]
1


3 ,
4
3 /A
1
X
Q
1
1
1
]
1



4 . /
. 4 .
/ . /4
.
1
1
X
Q
( )
1
1
1
]
1






3 22 3
22 , 2
3 2 3
1/
1 1
1
X
Q
and
( ) [ ]
%
Q
Q . 11 , . A , . /
1 1
1
1 /

@ontin#ing for two more iterations, we obtain


e. = G4.2..., /.2, ..2..H
%
e, = G4.2..4, /.-33, ..2..H
%
As the components of e have stabili7ed to three significant fig#res, we ta)e x1
L
= 4.2.., x/
L
= /.24, and V1
L
= ..2. with
!
L
= /x1
L
9 x1
L
x/
L
9 . x/
L
= 14.1
Qy e%pressing ! as a c#bic! f#nction of x1 alone, we can easily see that in this partic#lar case the Dewton&Kaphson method has
converged on a local ma%im#m.
,. :se the penalty f#nction approach to
ma%imi7e: ! = M , M .1 M x1!
/
M 1 M x/!
/

s.t. .x1 9 x/ = -
Nenalty f#nction for this problem becomes
ma%imi7e: ( ) ( ) ( )
/
/ 1 1
/
/
/
1
- . 1 1 . , ] + x x 3 x x !
"his #nconstrained ma%imi7ation program in the two variables x1 and x/ is s#fficiently simple that it may be solved analytically.
(etting , 4 ] ! we obtain
( )
( )
1 / 1 1 1
1 / 1 1 1
- 1 1 .
- 1 . 1
3 x 3 x 3
3 x 3 x 3
+ + +
+ + +
(olving these e'#ations for x1 and x/ in terms of 31, we obtain
( )
( ) , 1
- 1
, 1
- 1
1
1
1
1
/ 1
+
+

+
+

3
3
3
3
x x
(ince the 6essian matri%
1
]
1




1 1
1 1
]
/ / 2
2 1A 2
3 3
3 3
!
is negative definite for every positive val#e of 31, !] is a strictly concave f#nction, and its sole stationary point m#st be a global
ma%im#m. "herefore, letting 31 _ 9W we obtain the optimal sol#tion to the original program:


1 1
,
-
x x


/ /
,
-
x x
with
( ) ( ) /- . , 1 1 . ,
/
/
/
1


x x !
-. :se the penalty f#nction approach to
minimi7e: ! = x1 M x/!
/
9 x. M 1!
/
91
s.t. x1
-
9 x/
-
9 x.
-
= 12
N#tting this program in standard form, we have
,1&-4
ma%imi7e: ! = M x1 M x/!
/
M x. M 1!
/
M 1 1!
s.t. x1
-
9 x/
-
9 x.
-
M 12 = 4
Ior program 1!, penalty f#nction becomes
ma%imi7e:
( ) ( ) ( )
/
-
.
-
/
-
1 1
/
.
/
/ 1
12 1 1 ] + + x x x 3 x x x !
/!
6,ase 1. <e set 31 = 4.4/ in /! and consider the program
ma%imi7e:
( ) ( ) ( )
/
-
.
-
/
-
1
/
.
/
/ 1
12 4/ . 4 1 1 ] + + x x x x x x !
.!
Arbitrarily selecting G4, 4, 4H
%
as o#r initial vector, and setting , = 1, we apply the modified pattern search to program .!. "he
res#lt after 1A f#nctional eval#ations is G1, 1, 1H
%
, with
f1, 1, 1! = M 1 and E11, 1, 1! = M 1.
6,ase 2. (ince E11, 1, 1! = M 1. ^ 4, the constraint in program 1! is not satisfied. "o improve this sit#ation, we increase 31 in /!
to 4./ and consider the program
ma%imi7e:
( ) ( ) ( )
/
-
.
-
/
-
1
/
.
/
/ 1
12 / . 4 1 1 ] + + x x x x x x !
,!
"a)ing G1, 1, 1H
%
from 6,ase1 as the initial appro%imation, we apply the modified pattern search to ,!, still )eeping , = 1. "he
res#lt remains G1, 1, 1H
%
, indicating that the constraint cannot be satisfied in integers.
6,ase 3. (ince increasing 31 did not improve the c#rrent sol#tion, we ret#rn to program .!, red#ce , to 4.1, and ma)e a new
pattern search, again with G1, 1, 1H
%
as initial appro%imation. "he res#lt is G1.-, 1.-, 1H
%
with
f1.-, 1.-, 1! = M 1 and E11.-, 1.-, 1! = 4.1A1-
@ontin#ing in this manner, we complete the table below. :sing the res#lts of 6,ase 3, we concl#de that x1
L
= 1.,32, x/
L
= 1.,32,
%.
L
= 1.44., with !
L
= 91.444, appro%imates the optimal sol#tion to the original minimi7ation program.
Nhase 31 , Iinal +ecgtor O fO! E1O!
x1 x/ x.
1 4.4/ 1 1 1 1 M 1 M 1.
/ 4./ 1 1 1 1 M 1 M 1.
. 4.4/ 4.1 1.- 1.- 1 M 1 4.1A1-
, 4./ 4.1 1.- 1.- 1 M 1 4.1A1-
- 4.4/ 4.41 1.,3 1.- 1 M 1.444 M 4.42/.
2 4./ 4.41 1.,3 1.- 1.41 M 1.444 M 4.411.
1 4./ 4.441 1.132 1.,32 1.44/ M 1.444 M 4.44.3
A / 4.441 1.,32 1.,32 1.44. M 1.444 4.441/
3 /4 4.441 1.,32 1.,32 1.44. M 1.444 4.441/
2. (olve the following program by #se of the S#hn&"#c)er conditions:
minimi7e:
. / 1 . / . 1 / 1
/
.
/
/
/
1
- 14 / 1/ 2 , 14 - x x x x x x x x x x x x ! + + + + +
s.t. x1 9 /x/ 9 x. ,
with: all variables nonnegative
Iirst transforming into the standard form and then introd#cing s'#ared slac) variables, we obtain
ma%imi7e:
. / 1 . / . 1 / 1
/
.
/
/
/
1
- 14 / 1/ 2 , 14 - x x x x x x x x x x x x ! + + +
s.t. M x1 M /x/ M x. 9 , 9 x,
/
= 4
M x1 9 x-
/
= 4
M x/ 9 x2
/
= 4
M x. 9 x1
/
= 4
Ior this program, the Lagrangian f#nction is
( ) ( ) ( ) ( )
/
1 . ,
/
2 / .
/
- 1 /
/
, . / 1 1
. / 1 . / . 1 / 1
/
.
/
/
/
1
, /
- 14 / 1/ 2 , 14 -
x x x x x x x x x x
x x x x x x x x x x x x Q
+ + + + +
+ + +

"a)ing the derivatives indicated in 3! and 14!, we have
,A&-4
11! 4
14! 4
3! 4
A! 4 , /
1! 4 /
2! 4 /
-! 4 /
,! 4 /
.! 4 - 1/ 2 /4
/! 4 / 14 1/ , 14
1! 4 / 2 , /
/
1 .
,
/
2 /
.
/
- 1
/
/
, . / 1
1
1 ,
1
2 .
2
- /
-
, 1
,
, 1 / 1 .
.
. 1 . 1 /
/
/ 1 . / 1
1

+ +

+ + +

+ + + +

+ + + +

x x
Q
x x
Q
x x
Q
x x x x
Q
x
x
Q
x
x
Q
x
x
Q
x
x
Q
x x x
x
Q
x x x
x
Q
x x x
x
Q




"hese e'#ations can be simplified. (et
s1 = x,
/
1/!
$'#ations ,! thro#gh 1! imply respectively that either V1 or x,, either V/ or x-, either V. or x2, either V, or x1, e'#als 7ero. Q#t, by
3! thro#gh 1/!, x,, x-, x2, and x1 are 7ero if and only if s1, x1, x/, and x, are respectively 7ero. "h#s, ,! thro#gh 1! and 3! thro#gh
1/! are e'#ivalent to the system
4
4
4
4
. ,
/ .
1 /
1 1

x
x
x
s

1.!
"here are 12 sol#tions to this system. Kes#lts are listed in the table below
V1 V2 V3 V4 x1 x2 x3 s1
1 4 4 4 4 11.- &. &-.- &,
/ 4 4 4 11 &- &. 4 &1-
. 4 4 2 4 11.- 4 &-.- &,
, 4 4 2 11 1 4 4 &.
- 4 &1.2,. 4 4 4 &,.2,. &..4.2 &12../
2 4 / 4 11 4 &1 4 &/
1 4 &..- 1. 4 4 4 &4./- &,./-
A 4 &/ 14 - 4 4 4 &,
3
4..A4
3 4 4 4
1,..
2 &/./.A &-.AA1 4
1
4 1.12, 4 4
1,.-
.
/.3,
1
4.-/3
, 4 4
1
1 &../ 4 1A.A 4 2.. 4 &/.. 4
1
/ 2 4 &A 11 , 4 4 4
1
. 2.2/. &A.1.A 4 4 4 1.-41
4.3A-
- 4
1
, 1- &/- 4 &., 4 / 4 4
1
- A- &2. &/4A 4 4 4 , 4
,3&-4
1
2 J J J J 4 4 4 4
"he only row in this table having nonnegative entries for all variables, as re'#ired by the S#hn&"#c)er conditions, is row 14 with
!
L
= ../.-.
-4&-4