620
620


362
362
Applied Operations Research
Applied Operations Research
Column Generation
Column Generation
Department of Mathematics and Statistics
Department of Mathematics and Statistics
The University of Melbourne
The University of Melbourne
This presentation has been made in accordance with the provision This presentation has been made in accordance with the provisions of Part VB of the copyright s of Part VB of the copyright
act for the teaching purposes of the University. act for the teaching purposes of the University.
For use of students of the University of Melbourne enrolled in t For use of students of the University of Melbourne enrolled in the subject 620 he subject 620 362. 362.
Copyright Copyright
2008 by 2008 by Heng Heng Soon Soon Gan Gan
Some contents of this presentation are adapted from year 2005 co Some contents of this presentation are adapted from year 2005 course notes for 620 urse notes for 620 362 362
Applied Operations Research, Department of Mathematics and Stati Applied Operations Research, Department of Mathematics and Statistics, The University of stics, The University of
Melbourne (compiled by Prof Natashia Boland and Dr Renata Melbourne (compiled by Prof Natashia Boland and Dr Renata Sotirov Sotirov) )
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 2
Column Generation
Column Generation
Why is column generation used?
Why is column generation used?
LPs with very large numbers of variables need to be solved. LPs with very large numbers of variables need to be solved.
Why formulate problems with large numbers of (integer)
Why formulate problems with large numbers of (integer)
variables?
variables?
The The bounds bounds provided by their LP provided by their LP relaxations are much relaxations are much
better better than those of alternative compact formulations, . than those of alternative compact formulations, .e.g e.g
the cutting stock gap is well the cutting stock gap is well known to be known to be 1 generally, so 1 generally, so
very little branching is required to get integer optimum. very little branching is required to get integer optimum.
Symmetry is eliminated or reduced Symmetry is eliminated or reduced as compared to as compared to
compact formulations, leading to much smaller B&B trees. compact formulations, leading to much smaller B&B trees.
Complex constraints Complex constraints, e.g. regulations specified in , e.g. regulations specified in
Enterprise Bargaining Agreement (EBA) can be relegated to Enterprise Bargaining Agreement (EBA) can be relegated to
a a subproblem subproblem, and solved by specialised techniques. , and solved by specialised techniques.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 3
What is column generation?
What is column generation?
A method for solving LPs in which
A method for solving LPs in which
there are very large number of
there are very large number of
variables, without having to enumerate
variables, without having to enumerate
all the variables
all the variables
a priori
a priori
.
.
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 4
Where has column generation been applied?
Where has column generation been applied?
Multi
Multi


commodity flow problems
commodity flow problems
Crew rostering
Crew rostering
Vehicle routing
Vehicle routing
x
i b x a
t s
x
i
P p
p ip
P p
p
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 11
A Cutting Stock Problem
A Cutting Stock Problem
Step 0
Step 0
:
:
Choose subset of columns corresponding to
Choose subset of columns corresponding to
cutting patterns (5 0 0)
cutting patterns (5 0 0)
T T
, (0 3 0)
, (0 3 0)
T T
, (0 0 1)
, (0 0 1)
T T
, i.e.
, i.e.
(
(
(
=
1 0 0
0 3 0
0 0 5
s
A
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 12
A Cutting Stock Problem
A Cutting Stock Problem
Step 1:
Step 1:
Solve
Solve
Optimal basic solution is
Optimal basic solution is
y
y
1 1
= 5, y
= 5, y
2 2
= 6 2/3, y
= 6 2/3, y
3 3
= 15
= 15
and optimal dual variables
and optimal dual variables
u = (1/5, 1/3, 1)
u = (1/5, 1/3, 1)
+ +
0
15
20 3
25 5
. .
min
: LP Restricted
3
2
1
3 2 1
y
y
y
y
t s
y y y
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 13
A Cutting Stock Problem
A Cutting Stock Problem
Step 2:
Step 2:
Solve column generation
Solve column generation
subproblem
subproblem
:
:
But what does it mean for
But what does it mean for
a
a
p p
to be a column of
to be a column of
the Master LP?
the Master LP?
a
a
p p
= (a
= (a
1p 1p
, a
, a
2p 2p
, a
, a
3p 3p
)
)
T T
is integer,
is integer,
0 and
0 and
satisfies
satisfies
3a
3a
1p 1p
+ 5a
+ 5a
2p 2p
+ 9a
+ 9a
3p 3p
17
17
( )
LP Master the of column a is
. .
1 , 3 / 1 , 5 / 1 1 min
p
p
a
t s
a
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 14
A Cutting Stock Problem
A Cutting Stock Problem
Therefore column generation
Therefore column generation
subproblem
subproblem
is
is
+ +
+ +
=
+ +

\

+ +
=
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
max
1
integer , 0 , ,
17 9 5 3
. .
3
1
5
1
1 min
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
a a a
a a a
t s
a a a
z
Optimal solution:
a
1p
= a
2p
= a
3p
= 1
z
sub
= 1 (1/5 + 1/3 + 1)
= 8/15 < 0
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 15
A Cutting Stock Problem
A Cutting Stock Problem
Minimum reduced cost is Minimum reduced cost is negative negative, so column , so column
(1 1 1) (1 1 1)
T T
should be added to A should be added to A
s s
. .
We return to Step 1 and solve the new restricted LP: We return to Step 1 and solve the new restricted LP:
Optimal basic solution is Optimal basic solution is
y y
1 1
= 2, y = 2, y
2 2
= 1 2/3, y = 1 2/3, y
3 3
= 0, y = 0, y
4 4
= 15 = 15
and optimal dual variables and optimal dual variables
u = (1/5, 1/3, 7/15) u = (1/5, 1/3, 7/15)
+
+
+
+ + +
0
15
20 3
25 5
. .
min
: LP Restricted
4 3
4 2
4 1
4 3 2 1
y
y y
y y
y y
t s
y y y y
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 16
A Cutting Stock Problem
A Cutting Stock Problem
Column generation
Column generation
subproblem
subproblem
is
is
Optimal solution to
Optimal solution to
subproblem
subproblem
is
is
a
a
1 1
= a
= a
2 2
= a
= a
3 3
= 1
= 1
Therefore
Therefore
z
z
sub sub
= 1
= 1
(1/5+1/3+7/15) = 0
(1/5+1/3+7/15) = 0
+ +
+ +
=
integer , 0 , ,
17 9 5 3
. .
15
7
3
1
5
1
max
1
3 2 1
3 2 1
3 2 1
p p p
p p p
p p p
sub
a a a
a a a
t s
a a a
z
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 17
A Cutting Stock Problem
A Cutting Stock Problem
Thus there can be no negative reduced cost columns
Thus there can be no negative reduced cost columns
and optimal solution to Master LP is to use
and optimal solution to Master LP is to use
2 x (5 0 0)
2 x (5 0 0)
T T
,
,
1 2/3 x (0 3 0)
1 2/3 x (0 3 0)
T T
,
,
15 x (1 1 1)
15 x (1 1 1)
T T
Note:
Note:
This is not integer!! This is not integer!!
But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on But LP value = 2 + 1 2/3 + 15 = 18 2/3 is lower bound on
problem. problem.
Observation: Observation: Now Now
2 x (5 0 0) 2 x (5 0 0)
T T
, 2 x (0 3 0) , 2 x (0 3 0)
T T
, 15 x (1 1 1) , 15 x (1 1 1)
T T
is an integer feasible solution with value 19. is an integer feasible solution with value 19.
Is this the optimal solution? Is this the optimal solution?
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 18
A Cutting Stock Problem
A Cutting Stock Problem
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 19
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Tasks to be performed: 1, 2, Tasks to be performed: 1, 2, , n , n
h h
i i
= time required for task i (hours) = time required for task i (hours)
H = max number of hours worker can work in a day H = max number of hours worker can work in a day
What is the minimum workforce required? What is the minimum workforce required?
{ } J s x
n i x
t s
x z
s
s i J s
s
J s
s
= =
=
, 1 , 0
, , 1 , 1
. .
min
: IP Master
:
K
{ }
=
=
n i y
H y h
s.t.
y u
z
i
n
i
i i
n
i
i i
sub
, , 1 , 1 , 0
max
1
: Subproblem
1
1
K
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 20
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Solve Master LP
Solve subproblem
Add columns?
Yes
?
No
Initialise Columns
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 21
Human Resource Planning (HRP)
Human Resource Planning (HRP)
Sample
Sample
Xpress
Xpress
MP MP
implementation
implementation
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 22
Human Resource Planning (HRP)
Human Resource Planning (HRP)
1 (70) 1 (70)
33 33
33 33
384 384
50 50
1 (76) 1 (76)
23 23
23 23
390 390
40 40
HRP HRP
(Partition (Partition 
Heuristic) Heuristic)
33 33 33 33 33 33
(~7.8%, 60s) (~7.8%, 60s)
MIP value MIP value
23 23 23 23 23 23
MIP value MIP value
33 33 30.425 30.425 30.425 30.425
LP value LP value
1 (60) 1 (60)
50 50 colgen colgen iters iters
141 (530) 141 (530) 63,023 (242) 63,023 (242)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
83 83 1,734 1,734 1,734 1,734
# variables # variables
50 50 117 117 84 84
# constraints # constraints
50 tasks 50 tasks
1 (46) 1 (46)
48 48 colgen colgen iters iters
1,079 (443) 1,079 (443) 471 (204) 471 (204)
# nodes # nodes
(# simplex iterations) (# simplex iterations)
23 23 22.25 22.25 22.25 22.25
LP value LP value
71 71 984 984 984 984
# variables # variables
40 40 87 87 64 64
# constraints # constraints
40 tasks 40 tasks
HRP HRP
(Partition (Partition 
ColGen ColGen) )
HRP HRP
(Assignment (Assignment  
SymBreak SymBreak) )
HRP HRP
(Assignment) (Assignment)
Scenario
Scenario
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 23
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 24
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 25
A Vehicle Routing Problem (VRP)
A Vehicle Routing Problem (VRP)
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 26
An Air Crew Scheduling Problem
An Air Crew Scheduling Problem
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 27
An Air Crew Scheduling Problem
An Air Crew Scheduling Problem
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 28
Further reading
Further reading
Winston Chapter 10.3
Winston Chapter 10.3
620362 Applied Operations Research (Department of Mathematics & Statistics, University of Melbourne) 29
Next lecture
Next lecture