MIE365 ORIII Lecture Notes

Lecture 1a: Column Generation Development

This is a new method to solve a classical OR problem – Cutting Stock Problem - that leads to a totally
new expansion of the capabilities of linear programming.

Suppose you sell boards of lumber to the retail market in lengths of 3’, 5’ and 9’, to meet a demand of

• 25 units of 3’ boards
• 20 units of 5’ boards
• 15 units of 9’ boards

Suppose also that you can only buy boards in the wholesale market in 17’ lengths. How many 17’ lengths
do we need and how should we cut them to meet the demand at minimum waste.

Unfortunately, we cannot simply start to define decision variables to formulate this problem into a
mathematical model. We need some preliminary analysis.

It turns out that one of the issues we have to deal with is to define in advance all the possible ways that
we can cut a 17’ board into one or more of the target lengths.

This is best done methodically by using a table of all possible cuts:

Cut Type 1 2 3 4 5 6
3’ 5 4 2 2 1 0
5’ 0 1 2 0 1 3
9’ 0 0 0 1 1 0
waste 2 0 1 2 0 2

Note that we have omitted any cut which results in a waste ≥ 3’.

Now the problem can be formulated as follows:

Let xi be the number of boards cut according to cut type i. and z the overall waste.

Then we might formulate the following LP problem:

Min z = 2x1 + x3 + 2x4 + 2x6

subject to

5x1 + 4x2 + 2x3 + 2x4 + x5 ≥ 25

x2 +2x3 +x5 + 3x6 ≥ 20

x4 + x5 ≥ 15

All xi ≥ 0
Daniel Frances  2017 1
MIE365 ORIII Lecture Notes

While it would seem that we would need to restrict the variables to integers, in practice rounding off
works quite well for this type of problem, we are done, or are we?

Two remaining problems:

• Is the waste expression correct? If we exceed demand is that not also waste?

• What if instead of 17’ boards, we were dealing with 107’ wire lengths to be cut into 3’, 5’ and 9’
lengths – the number of possible cuts would render this method impractical.

A 2nd look at the objective

In addition to the waste that we cannot sell because there is no demand for the “wrong” size, we also need
to account for the waste that we cannot sell because we are producing too many units of the “right” size.

The new objective which includes both types of waste becomes

z = 2x1 + x3 + 2x4 + 2x6

+ 3*(5x1 + 4x2 + 2x3 + 2x4 + x5 – 25)
+ 5*(x2 + 2x3 + x5 + 3x6 – 20)
+ 9*(x4 + x5 – 15)

= 17x1 + 17x2 + 17x3 + 17x4 + 17x5 + 17x6 – 310

Clearly this objective can be replaced by the simpler

z = x1 + x2 + x3 + x4 + x5 + x6  minimize the total number of 17’ lengths cut.

Thus the new LP becomes

Min z = x1 + x2 + x3 + x4 + x5 + x6

subject to

5x1 + 4x2 + 2x3 + 2x4 + x5 ≥ 25

x2 +2x3 +x5 + 3x6 ≥ 20

x4 + x5 ≥ 15

All xi ≥ 0

Daniel Frances  2017 2

MIE365 ORIII Lecture Notes

What to do when the number of cut types is too large to be listed

This is where it gets interesting!

Thus suppose that the material comes in 107’ lengths

Let’s start to build the equivalent of the previous table.

Cut Type 1 2 3 4 ...

3’ 35 34 32 31 ...
5’ 0 1 2 1 ...
9’ 0 0 0 1 ...
waste 2 0 1 0 ...

Is it really necessary to complete this table, or is there a better way?

Designer Cuts

With this approach we first select a number of start-up cuts with which we can meet the demand, say cut
1 which only produces 3’ lengths, cut 2 which only produces 5’lengths and cut 3 which only produces 9’
lengths. then design a “new” cut that is better than the existing cuts by solving an IP problem.

Below is the table for this approach. Cut 4 is the new better cut we are designing, and we let a, b and c, be
the number of 3’, 5’ and 9’ lengths it should have.

Cut Type 1 2 3 4
3’ 35 0 0 a
5’ 0 21 0 b
9’ 0 0 11 c

Since we initially use cuts 1, 2, and 3, then 𝑥𝑥1 , 𝑥𝑥2 , 𝑥𝑥3 must be in the basis, and so far 𝑥𝑥4 is zero and non-
basic. We would like to choose a, b, c, in such a way that the simplex method will definitely use cut 4,
and bring it into the basis, and drop one of the existing cuts, by removing it from the basis.

If you recall the criterion that simplex uses for a variable to enter the basis are the entries in the z-
equation. Thus the IP starts to look like

Max or Min coefficient of 𝑥𝑥4 of the z-equation (in terms of a, b, c)

Subject to 3𝑎𝑎 + 5𝑏𝑏 + 9𝑐𝑐 ≤ 107, 𝑎𝑎, 𝑏𝑏, 𝑐𝑐 ≥ 0, 𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖

At this point we have to find a way of building the objective of the IP, and will use LP matrix notation,
and relax the requirement that the 𝒙𝒙𝒊𝒊 have to be integer.

Daniel Frances  2017 3

MIE365 ORIII Lecture Notes

In non-matrix notation the LP has the following relations

𝑧𝑧 = 𝑥𝑥1 + 𝑥𝑥2 + 𝑥𝑥3 + 𝑥𝑥4 subject to

35𝑥𝑥1 + 𝑎𝑎𝑥𝑥4 − 𝑠𝑠1 = 25

21𝑥𝑥2 + 𝑏𝑏𝑥𝑥4 − 𝑠𝑠2 = 20

11𝑥𝑥3 + 𝑐𝑐𝑥𝑥4 − 𝑠𝑠3 = 15

Thus in matrix notation the relations are 𝑧𝑧 = 𝑐𝑐 𝑇𝑇 𝑥𝑥 and 𝐴𝐴𝐴𝐴 = 𝑏𝑏, 𝑥𝑥 ≥ 0

𝑥𝑥1 1
𝑥𝑥2 1
⎛𝑥𝑥3 ⎞ ⎛ ⎞  25 
1 35 0 0 𝑎𝑎 −1 0 0  
⎜ ⎟ ⎜ ⎟
where 𝑥𝑥 = ⎜𝑥𝑥4 ⎟ , 𝑐𝑐 = ⎜1⎟ , 𝐴𝐴 = � 0 21 0 𝑏𝑏 0 −1 0 � b =  20 
⎜ 𝑠𝑠1 ⎟ ⎜0⎟ 0 0 11 𝑐𝑐 0 0 −1  
𝑠𝑠2  15 
0
⎝ 𝑠𝑠3 ⎠ ⎝0⎠

Since we are using only cuts 1, 2 and 3, the corresponding variables 𝑥𝑥1 , 𝑥𝑥2 , 𝑥𝑥3 will be basic, and the
remaining variables non-basic. Then we separate all the vectors into the basic and non-basic parts.

Thus z = 𝑐𝑐𝐵𝐵𝑇𝑇 𝑥𝑥𝐵𝐵 + 𝑐𝑐𝑁𝑁𝑇𝑇 𝑥𝑥𝑁𝑁 and 𝐵𝐵𝑥𝑥𝐵𝐵 + 𝑁𝑁𝑥𝑥𝑁𝑁 = 𝑏𝑏

𝑥𝑥4 1  31 0 0 
𝑥𝑥1 1 𝑎𝑎 −1 0 0
𝑠𝑠1 0  
Where 𝑥𝑥𝐵𝐵 = �𝑥𝑥2 � , 𝑥𝑥𝑁𝑁 = � 𝑠𝑠 �, 𝑐𝑐𝐵𝐵 = �1� , 𝑐𝑐𝑁𝑁 = � � B =  0 21 0  𝑁𝑁 = �𝑏𝑏 0 −1 0 �
𝑥𝑥3 2 0  0 0 11
𝑠𝑠3 1 𝑐𝑐 0 0 −1
0  

In LP terms, how do we know this basis is optimal? We need to check the signs of the z-equation to see
if there is any benefit to be had in entering a non-basic variable.

To see if it’s worthwhile introducing 𝑥𝑥4 into the basis we need to find a way for calculating the
coefficients of 𝑥𝑥4 in the z-equation.

Rearrange the constraint 𝐵𝐵𝑥𝑥𝐵𝐵 + 𝑁𝑁𝑥𝑥𝑁𝑁 = 𝑏𝑏 as 𝑥𝑥𝐵𝐵 = 𝐵𝐵 −1 𝑏𝑏 − 𝐵𝐵 −1 𝑁𝑁𝑥𝑥𝑁𝑁

Then substitute into 𝑧𝑧 = 𝑐𝑐𝐵𝐵𝑇𝑇 (𝐵𝐵−1 𝑏𝑏 − 𝐵𝐵 −1 𝑁𝑁𝑥𝑥𝑁𝑁 ) + 𝑐𝑐𝑁𝑁𝑇𝑇 𝑥𝑥𝑁𝑁 = 𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1 𝑏𝑏 − (𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵−1 𝑁𝑁 − 𝑐𝑐𝑁𝑁𝑇𝑇 )𝑥𝑥𝑁𝑁

Thus for minimization of the LP, we should maximize the entry in (𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1 𝑁𝑁 − 𝑐𝑐𝑁𝑁𝑇𝑇 ) corresponding to 𝑥𝑥4 ,
1/35 0 0 𝑎𝑎
1 1 1
𝑀𝑀𝑀𝑀𝑀𝑀 (1 1 1) � 0 1/21 0 � �𝑏𝑏 � − 1 = � � 𝑎𝑎 + � � 𝑏𝑏 + � � 𝑐𝑐 − 1
35 21 11
0 0 1/11 𝑐𝑐

Daniel Frances  2017 4

MIE365 ORIII Lecture Notes

Thus the Designer Cut is obtained by solving the LP

1 1 1
𝑀𝑀𝑀𝑀𝑀𝑀 � � 𝑎𝑎 + � � 𝑏𝑏 + � � 𝑐𝑐 − 1 subject to 3𝑎𝑎 + 5𝑏𝑏 + 9𝑐𝑐 ≤ 107, 𝑎𝑎, 𝑏𝑏, 𝑐𝑐 ≥ 0, 𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖
35 21 11

Is there a more efficient way of building the objective function of the IP?

Note that the coefficient of a, b, c come from the expression 𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1 .

When we use a computer to solve an LP, is there an efficient way of extracting 𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1.?

Recall that the dual objective is given as 𝑤𝑤 = 𝑏𝑏 𝑡𝑡 𝑦𝑦, where y are the dual variables, which are also the
shadow prices of the primal, and that at optimality 𝑧𝑧 = 𝑤𝑤, so that 𝑧𝑧 = 𝑏𝑏 𝑡𝑡 𝑦𝑦 = 𝑦𝑦 𝑡𝑡 𝑏𝑏

Note that the value of the objective at any iteration is 𝑧𝑧 = 𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1 𝑏𝑏. Comparing this to 𝑧𝑧 = 𝑦𝑦 𝑡𝑡 𝑏𝑏, we
conclude that 𝑐𝑐𝐵𝐵𝑇𝑇 𝐵𝐵 −1 is the vector of shadow prices, which can easily be extracted from an LP.

The Column Generation algorithm

1. Create simple cuts that satisfy the demand, and formulate the corresponding LP

2. Solve the LP and cut-and-paste the shadow prices for the constraints into the IP objective

3. Solve the IP and cut and paste the new cut into the LP

4. Return to Step 2, as long as the optimal IP objective is positive

5. Solve the LP with all the cuts, as an IP enforcing integer 𝑥𝑥𝑖𝑖 values.

Next, a demo for our example.

Daniel Frances  2017 5