ISM206 Optimization

Scribing Notes

LECTURE: INTRO TO LINEAR PROGRAMMING AND THE SIMPLEX METHOD, KEVIN ROSS
MARCH 31, 2005

DAVID L. BERNICK dbernick@soe.ucsc.edu

1. Overview Typical Linear Programming problems Standard form and converting to Standard form Geometry of Linear Programming Extreme points, linear independence and bases Optimality conditions The Simplex method

2. Typical Linear programming Problems 2.1. Product Mix problem Problem: How much beer and ale should be produced? amt. item(pounds) 480 corn available resources = 160 hops 1190 malt item corn hops malt (barrel) (pounds) (ounces) (pounds) required resources = Ale 5 4 35 15 4 20 Beer Ale $13/barrel Beer $23/barrel
1

(1)

(2)

prof it =

(3)

ISM206 Optimization

Scribing Notes

Analysis:

A = Ale(barrels) B = Beer(barrels) Objective f unction M ax{P rof it = 13A + 23B } Constraints : Subject to(s.t) 5A + 15B 480 4A + 4B 160 35A + 20B 1190 A, B 0

In general:

ci = producti prof it xi = producti count bj = resourcej availability ai,j = resourcej required f or producti objective f unction = max{
i{products}

ci xi }

s.t.
i{products}

ai,j xi bj xi 0

2.2. Transportation Problem Problem: Production of computers in Singapore and Oakland Distribution centers in Oakland, Hong Kong and Istanbul

ISM206 Optimization

Scribing Notes

Supply, demand and cost summary: ship.cost Oakland HongKong Istanbul Supply 85 37 119 500 Singapore Supply, demand, cost = Hohboken 53 189 94 300 Demand 350 250 200 (4) Objective: meet demand with minimum total cost. Analysis: ci,j = ship.cost f rom i to j Si = supply f rom sitei Dj = demand f rom sitej i {Singapore, Hohboken} j {Oakland, HongKong, Istanbul} xi,j = count shipped f rom i to j objective =
i,j

ci,j xi,j

s.t.
j

xi,j = Si i(supply ) xi,j = Dj j (demand)

i

xi 0 3. Other LP examples Blending Problem Diet Problem Assignment Problem 3.1. Blending Problem Problem: Consider a constraint where at least 10% of output must be from 1 variable. We then have a constraint of the form: xixi 0.10. This is not a i linear equation but we can change it to: xi 0.10 i xi and now it is a linear form.

ISM206 Optimization

Scribing Notes

4. Key elements of a LP Proportionality - the function depends on a proportion assigned to a variable. Additivity Divisibility 4.1. Steps in building a LP Identify the activities. Identify the items. Identify the Input / Output coecients. Write the constraints. Identify the coecients of the objective function.

5. Geometry of a LP Consider the plot of the solution space for the following: objective f unction : max{x1 + x2 } s.t. 3x1 5x2 15 3x1 5x2 12 x1 , x2 0 As a quick way to nd the regions specicd by the inequality, consider weather (0,0) is a solution for the inequality or not. In this case, (0,0) is part of the solution spacec for all constraints. Geometrically, this problem can be viewed as Fig:??. As can be seen, the solution is at a corner. This is true in general for Linear Programming problems - one of the optimal points will be at a corner. This is the basis of the Simplex method. 6. Standard Form Types of LP descriptors: Objective f unction : max(c1 x1 + c2 x2 + . . . cN xN s.t a1,1 x1 + a1,2 x2 + . . . + a1,N xN = b1 a2,1 x1 + a2,2 x2 + . . . + a2,N xN = b2 xj 0, j = 1, N

ISM206 Optimization

Scribing Notes

Figure 1. Graph of feasible region, bounded by all constraints, with family of curves of x1 + x2 and max(x1 + x2 ).

or consisely: max(c x) s.t. Ax = b x0

where A is an [m x n] matrix of N variables and m constraints. All LP problems can be converted to standard form. The following situations can arise, and are converted to standard form as follows: inequalities free variables MINimizations

6.1. Handling Inequalities For example, the constraint x1 + x2 4 can be handled by introducing a new, surplus variable x3 . This constraint then becomes x1 + x2 + x3 = 4 s.t x3 0.

ISM206 Optimization

Scribing Notes

6.2. Handling Free variables In the case where a variable is free (x1 is unconstrained), then we once again introduce a new, surplus variable. For example: x1 objective f unction : min(x1 + x2 ) s.t. x2 0 x1 + x2 = 3
replace x1 = x+ 1 x1 x+ 1 , x1 0

(f ree variable)

rewritten in standard form: objective f unction : min x1 + x2 x+ 1 + s.t. x2 x1 + x2 x+ 1 , x1 new variable does not appear in the objective function 0 =3 0

6.3. Handling MIN objectives (vs. MAX) To convert a MIN to a MAX, negate the terms. For example: min x1 + 3x2 = max x1 3x2 . Remember, any surplus variable that is introduced does not appear in the objective function - only in constraints.

7. Solutions, Extreme points and Basis How many solutions are there to a set of linear equations? Convexity of a feasible region. Extreme Points

ISM206 Optimization

Scribing Notes

7.1. How many solutions exist for a set of linear equations How many linearly independent rows and columns exist in the set? In matrix notation, the set of equations is described by Ax = b . If A is a square matrix (NxN) and det(A) = 0 then x = Ab is unique. In general, A is rectangular (m x N), with many variable and few constraints. 7.2. Convexity of a feasible region

X a convex set i x1 , x2 X x1 + (1 )x2 X , {0 1} or, for any two points in set X , any point between any point between those 2 points is also in the set. 7.3. Extreme Point of X x is an extreme point i f or distinct x1 , x2 X if, x = x1 + (1 )x2 , {0 1} = {0, 1} then x is an extreme point or, if x is not between 2 other points in the set, then it is an extreme point. 8. Linear independence of vectors Basis of a matrix A basic solution of an LP Basic Feasible solution (Corner Point Feasible) 8.1. Basic Feasible solution(bfs) x is an extreme point of a solution space i it is a bfs of Ax = b, {x 0}. Key fact: If an LP has an optimal solution, it exists at a corner of the feasible region.

ISM206 Optimization

Scribing Notes

8.2. Basis of a matrix Linear independence

Let V 1 , V 2 , . . . , V N be vectors. They are linearly independent,

if : 1 V 1 + 2 V 2 . . . + N V N = 0 = 1 = 2 = . . . N = 0

Basis of matrix A is a maximal linearly independent set of columns of A. For example:

A=

110 111

(constraints)

Basis(A) = 1st and 3rd columns of A, or 2nd and 3rd columns

Only a certain number of constraints need to be binding.

8.3. Rank of a Matrix Rank of a matrix = number of independent columns in the matrix. [m x n] matrix A is full rank if rank(A)=m. Or, if there are at least as many idependent columns as there are constraints. a1,1 a1,2 . . . a1, N . A= . . am,1 am,2 . . . am,n

for m constraints and n variables.

ISM206 Optimization

Scribing Notes

8.4. Linear Programming Write A = [B, N ] where B is a basis of A. A= Basis of A = B = Lef tover or N ull = N = max cT x s.t. Ax = b x0 A = [B, N ] Rewrite constraints as: [B, N ] xB =b xN set xN = 0 xB = B 1 b BxB + N xN = b Set N xN = 0 XB = b XB = B 1 b 9. Simplex Method - Overview Checks Corner Points Looks for better solutions at each iteration Simplex Algorithm: Find a starting point - a corner. Test this point for optimality Stop if this point is optimal, otherwise repeat One basic variable is replaced by another. Optimality test identies the basic variable to replace. 12011 00111 10 01 211 011