You are on page 1of 29

620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 1

620
620
-
-
362
362
Applied Operations Research
Applied Operations Research
Column Generation
Column Generation
Department of Mathematics and Statistics
Department of Mathematics and Statistics
The University of Melbourne
The University of Melbourne
This presentation has been made in accordance with the provision This presentation has been made in accordance with the provisions of Part VB of the copyright s of Part VB of the copyright
act for the teaching purposes of the University. act for the teaching purposes of the University.
For use of students of the University of Melbourne enrolled in t For use of students of the University of Melbourne enrolled in the subject 620 he subject 620- -362. 362.
Copyright Copyright

2008 by 2008 by Heng Heng- -Soon Soon Gan Gan
Some contents of this presentation are adapted from year 2005 co Some contents of this presentation are adapted from year 2005 course notes for 620 urse notes for 620- -362 362
Applied Operations Research, Department of Mathematics and Stati Applied Operations Research, Department of Mathematics and Statistics, The University of stics, The University of
Melbourne (compiled by Prof Natashia Boland and Dr Renata Melbourne (compiled by Prof Natashia Boland and Dr Renata Sotirov Sotirov) )
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 2
Column Generation
Column Generation
Why is column generation used?
Why is column generation used?
LPs with very large numbers of variables need to be solved. LPs with very large numbers of variables need to be solved.
Why formulate problems with large numbers of (integer)
Why formulate problems with large numbers of (integer)
variables?
variables?
The The bounds bounds provided by their LP provided by their LP- -relaxations are much relaxations are much
better better than those of alternative compact formulations, . than those of alternative compact formulations, .e.g e.g
the cutting stock gap is well the cutting stock gap is well- -known to be known to be 1 generally, so 1 generally, so
very little branching is required to get integer optimum. very little branching is required to get integer optimum.
Symmetry is eliminated or reduced Symmetry is eliminated or reduced as compared to as compared to
compact formulations, leading to much smaller B&B trees. compact formulations, leading to much smaller B&B trees.
Complex constraints Complex constraints, e.g. regulations specified in , e.g. regulations specified in
Enterprise Bargaining Agreement (EBA) can be relegated to Enterprise Bargaining Agreement (EBA) can be relegated to
a a subproblem subproblem, and solved by specialised techniques. , and solved by specialised techniques.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 3
What is column generation?
What is column generation?
A method for solving LPs in which
A method for solving LPs in which
there are very large number of
there are very large number of
variables, without having to enumerate
variables, without having to enumerate
all the variables
all the variables
a priori
a priori
.
.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 4
Where has column generation been applied?
Where has column generation been applied?

Multi
Multi
-
-
commodity flow problems
commodity flow problems

Cutting stock problems


Cutting stock problems

Binary cutting stock problems


Binary cutting stock problems

Air crew scheduling


Air crew scheduling

Crew rostering
Crew rostering

Aircraft fleeting and routing


Aircraft fleeting and routing

Vehicle routing
Vehicle routing

Optical telecommunications design


Optical telecommunications design

Ship scheduling and routing


Ship scheduling and routing
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 5
A Quick revision on LP
A Quick revision on LP
1. 1. LP: LP: z z
lp lp
= = min{cx min{cx: : Ax Ax b, x b, x 0, 0, x x R R
n n
} }
2. 2. Very large LPS can be solved very efficiently with today Very large LPS can be solved very efficiently with today s s
software, e.g. software, e.g. Cplex Cplex, , Xpress Xpress
MP MP
. .
Usual method is still simplex method. Usual method is still simplex method.
3. 3. Dual LP: w = Dual LP: w = max{ub max{ub: : uA uA c, c, u u 0, 0, u u R R
m m
} }
4. 4. Simplex method maintains basis B for column space of A via Simplex method maintains basis B for column space of A via
subset of columns of A, indexed by B subset of columns of A, indexed by B {1, {1, ,n}. ,n}.
It maintains feasible x and dual variable u It maintains feasible x and dual variable u 0 with 0 with ub ub = = cx cx and strives and strives
to achieve to achieve uA uA c. c.
Simplex iteration exchanges two columns in basis, i.e. B:=B Simplex iteration exchanges two columns in basis, i.e. B:=B\ \{ {i}U{j i}U{j}, },
i i B B, , j j B B
To choose j: find most infeasible dual constraint, i.e. select To choose j: find most infeasible dual constraint, i.e. select
j j arg arg min(c min(c
k k
- - ua ua
k k
) < 0 ) < 0
where (c where (c
k k
- - ua ua
k k
) is the ) is the reduced cost reduced cost of of x x
k k
. .
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 6
Column Generation
Column Generation
What if A has a huge number of columns? How can we
What if A has a huge number of columns? How can we
solve the LP?
solve the LP?
Definition:
Definition:
We call the LP with a huge number of
We call the LP with a huge number of
columns the
columns the
Master LP
Master LP
.
.
Definition:
Definition:
If A
If A
s s
is a matrix formed from a subset of the
is a matrix formed from a subset of the
columns of A, and
columns of A, and
c
c
s s
is the corresponding
is the corresponding
subvector
subvector
of c, the
of c, the
LP LP
s s
: : z z
s s
LP LP
= = min{c min{c
s s
y y: : A A
s s
y y b, y b, y 0} 0}
is called the
is called the
Restricted LP
Restricted LP
.
.
Note: A
Note: A
s s
must be chosen so that LP is feasible and
must be chosen so that LP is feasible and
bounded below.
bounded below.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 7
Algorithm to solve the Master LP
Algorithm to solve the Master LP
Step 0: Step 0:
Choose an initial set of columns A Choose an initial set of columns A
s s
. .
Step 1: Step 1:
Solve the Restricted LP to obtain optimal basic solution Solve the Restricted LP to obtain optimal basic solution y y and and
corresponding duals corresponding duals u u. .
Step 2: Step 2:
Note that Note that x x given by given by
x x
i i
= = y y
i i
, if column , if column i i is in A is in A
s s
; 0, otherwise ; 0, otherwise
is a basic feasible point for the Master LP and is optimal if is a basic feasible point for the Master LP and is optimal if
uA uA c. c.
Solve column generation Solve column generation subproblem subproblem: :
z z
sub sub
= = min{c min{c
i i
ua ua
i i
: : a a
i i
is a column of A} is a column of A}
If If z z
sub sub
0 then STOP: 0 then STOP: x x is optimal for Master LP. is optimal for Master LP.
Otherwise add Otherwise add a a
i i
such that such that c c
i i
ua ua
i i
= = z z
sub sub
to A to A
s s
and go to Step 1. and go to Step 1.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 8
A Cutting Stock Problem
A Cutting Stock Problem
A cutting stock problem with 17m stock lengths
A cutting stock problem with 17m stock lengths
and with orders for 25 pieces of length 3m, 20
and with orders for 25 pieces of length 3m, 20
pieces of length 5m and 15 pieces of 9m.
pieces of length 5m and 15 pieces of 9m.
Let P be the set of all cutting patterns.
Let P be the set of all cutting patterns.
Let
Let
a
a
ip ip
denote the number of pieces of length
denote the number of pieces of length
l
l
i i
cut in pattern p, where l
cut in pattern p, where l
1 1
= 3, l
= 3, l
2 2
= 5 and l
= 5 and l
3 3
= 9.
= 9.
Let the demand for each length
Let the demand for each length
l
l
i i
be b
be b
i i
, where b
, where b
1 1
= 25, b
= 25, b
2 2
= 20, b
= 20, b
3 3
= 15.
= 15.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 9
A Cutting Stock Problem
A Cutting Stock Problem
a
1p
a
2p
a
3p
Pattern p
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 10
A Cutting Stock Problem
A Cutting Stock Problem
Variable:
Variable:
x
x
p p
= number of times pattern p is cut
= number of times pattern p is cut
Let A be the constraint matrix, i.e. A = (
Let A be the constraint matrix, i.e. A = (
a
a
ip ip
)
)
0
3 , 2 , 1 ,
. .
min

x
i b x a
t s
x
i
P p
p ip
P p
p
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 11
A Cutting Stock Problem
A Cutting Stock Problem
Step 0
Step 0
:
:
Choose subset of columns corresponding to
Choose subset of columns corresponding to
cutting patterns (5 0 0)
cutting patterns (5 0 0)
T T
, (0 3 0)
, (0 3 0)
T T
, (0 0 1)
, (0 0 1)
T T
, i.e.
, i.e.
(
(
(

=
1 0 0
0 3 0
0 0 5
s
A
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 12
A Cutting Stock Problem
A Cutting Stock Problem
Step 1:
Step 1:
Solve
Solve
Optimal basic solution is
Optimal basic solution is
y
y
1 1
= 5, y
= 5, y
2 2
= 6 2/3, y
= 6 2/3, y
3 3
= 15
= 15
and optimal dual variables
and optimal dual variables
u = (1/5, 1/3, 1)
u = (1/5, 1/3, 1)

+ +
0
15
20 3
25 5
. .
min
: LP Restricted
3
2
1
3 2 1
y
y
y
y
t s
y y y
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 13
A Cutting Stock Problem
A Cutting Stock Problem
Step 2:
Step 2:
Solve column generation
Solve column generation
subproblem
subproblem
:
:
But what does it mean for
But what does it mean for
a
a
p p
to be a column of
to be a column of
the Master LP?
the Master LP?
a
a
p p
= (a
= (a
1p 1p
, a
, a
2p 2p
, a
, a
3p 3p
)
)
T T
is integer,
is integer,

0 and
0 and
satisfies
satisfies
3a
3a
1p 1p
+ 5a
+ 5a
2p 2p
+ 9a
+ 9a
3p 3p

17
17
( )


LP Master the of column a is
. .
1 , 3 / 1 , 5 / 1 1 min
p
p
a
t s
a
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 14
A Cutting Stock Problem
A Cutting Stock Problem
Therefore column generation
Therefore column generation
subproblem
subproblem
is
is

+ +
+ +
=

+ +
|

\
|
+ +
=
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
max
1
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
1 min
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
a a a
a a a
t s
a a a
z
Optimal solution:
a
1p
= a
2p
= a
3p
= 1
z
sub
= 1 (1/5 + 1/3 + 1)
= -8/15 < 0
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 15
A Cutting Stock Problem
A Cutting Stock Problem
Minimum reduced cost is Minimum reduced cost is negative negative, so column , so column
(1 1 1) (1 1 1)
T T
should be added to A should be added to A
s s
. .
We return to Step 1 and solve the new restricted LP: We return to Step 1 and solve the new restricted LP:
Optimal basic solution is Optimal basic solution is
y y
1 1
= 2, y = 2, y
2 2
= 1 2/3, y = 1 2/3, y
3 3
= 0, y = 0, y
4 4
= 15 = 15
and optimal dual variables and optimal dual variables
u = (1/5, 1/3, 7/15) u = (1/5, 1/3, 7/15)

+
+
+
+ + +
0
15
20 3
25 5
. .
min
: LP Restricted
4 3
4 2
4 1
4 3 2 1
y
y y
y y
y y
t s
y y y y
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 16
A Cutting Stock Problem
A Cutting Stock Problem
Column generation
Column generation
subproblem
subproblem
is
is
Optimal solution to
Optimal solution to
subproblem
subproblem
is
is
a
a
1 1
= a
= a
2 2
= a
= a
3 3
= 1
= 1
Therefore
Therefore
z
z
sub sub
= 1
= 1

(1/5+1/3+7/15) = 0
(1/5+1/3+7/15) = 0

+ +
+ +
=
integer , 0 , ,
17 9 5 3
. .
15
7
3
1
5
1
max
1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
z
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 17
A Cutting Stock Problem
A Cutting Stock Problem
Thus there can be no negative reduced cost columns
Thus there can be no negative reduced cost columns
and optimal solution to Master LP is to use
and optimal solution to Master LP is to use
2 x (5 0 0)
2 x (5 0 0)
T T
,
,
1 2/3 x (0 3 0)
1 2/3 x (0 3 0)
T T
,
,
15 x (1 1 1)
15 x (1 1 1)
T T
Note:
Note:
This is not integer!! This is not integer!!
But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on
problem. problem.
Observation: Observation: Now Now
2 x (5 0 0) 2 x (5 0 0)
T T
, 2 x (0 3 0) , 2 x (0 3 0)
T T
, 15 x (1 1 1) , 15 x (1 1 1)
T T
is an integer feasible solution with value 19. is an integer feasible solution with value 19.
Is this the optimal solution? Is this the optimal solution?
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 18
A Cutting Stock Problem
A Cutting Stock Problem
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 19
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Tasks to be performed: 1, 2, Tasks to be performed: 1, 2, , n , n
h h
i i
= time required for task i (hours) = time required for task i (hours)
H = max number of hours worker can work in a day H = max number of hours worker can work in a day
What is the minimum workforce required? What is the minimum workforce required?
{ } J s x
n i x
t s
x z
s
s i J s
s
J s
s


= =
=



, 1 , 0
, , 1 , 1
. .
min
: IP Master
:
K
{ }

=
=
n i y
H y h
s.t.
y u
z
i
n
i
i i
n
i
i i
sub
, , 1 , 1 , 0
max
1
: Subproblem
1
1
K
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 20
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Solve Master LP
Solve subproblem
Add columns?
Yes
?
No
Initialise Columns
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 21
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 22
Human Resource Planning (HRP)
Human Resource Planning (HRP)
1 (70) 1 (70)
33 33
33 33
384 384
50 50
1 (76) 1 (76)
23 23
23 23
390 390
40 40
HRP HRP
(Partition (Partition- -
Heuristic) Heuristic)
33 33 33 33 33 33
(~7.8%, 60s) (~7.8%, 60s)
MIP value MIP value
23 23 23 23 23 23
MIP value MIP value
33 33 30.425 30.425 30.425 30.425
LP value LP value
1 (60) 1 (60)
50 50 colgen colgen iters iters
141 (530) 141 (530) 63,023 (242) 63,023 (242)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
83 83 1,734 1,734 1,734 1,734
# variables # variables
50 50 117 117 84 84
# constraints # constraints
50 tasks 50 tasks
1 (46) 1 (46)
48 48 colgen colgen iters iters
1,079 (443) 1,079 (443) 471 (204) 471 (204)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
23 23 22.25 22.25 22.25 22.25
LP value LP value
71 71 984 984 984 984
# variables # variables
40 40 87 87 64 64
# constraints # constraints
40 tasks 40 tasks
HRP HRP
(Partition (Partition- -
ColGen ColGen) )
HRP HRP
(Assignment (Assignment - -
SymBreak SymBreak) )
HRP HRP
(Assignment) (Assignment)
Scenario
Scenario
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 23
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 24
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 25
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 26
An Air Crew Scheduling Problem
An Air Crew Scheduling Problem
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 27
An Air Crew Scheduling Problem
An Air Crew Scheduling Problem
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 28
Further reading
Further reading
Winston Chapter 10.3
Winston Chapter 10.3
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 29
Next lecture
Next lecture

Column Generation Column Generation


& &
Branch Branch- -and and- -Bound Bound
Dantzig Dantzig- -Wolfe Decomposition Wolfe Decomposition