Beruflich Dokumente
Kultur Dokumente
Programming
(Pure & Binary)
Taufik Djatna, Dr.Eng.
Integer programming ::
Methods
Relaxation
Branch and Bound
Cutting planes
Column generation
Heuristics
Close the
Min Max Theorem gap!
Integer linear programming
Input:
A linear objective function f: Zn Z
A set of linear constraint functions: gi: Zn Z
A set of constraint values: bi
Goal:
Find x in Zn which:
1. maximizes f(x)
2. satisfies gi(x) <= bi
LIP Area of Application
Binary Integer Model:: Plant Expansion decision
Z=41.25
2 5x1 + 9x2 = 45
1
Z=20
0 1 2 3 4 5 6 7 8 9
x2
Utilizing the information about the
optimal solution of the LP-relaxation
Fact: If LP-relaxation has integral optimal solution x*,
then x* is optimal for IP too.
In our case, (x1, x2) = (2.25, 3.75) is the optimal solution of the LP-
relaxation. But, unfortunately, it is not integral.
3 (3, 3)
2
Subproblem 1
1
Z=20 Z=39
0 1 2 3 4 5 6 7 8
Solution tree
S1: x2 3
For each subproblem, we record
All
(3, 3) int.
the restriction that creates the subproblem Z=39
(2.25, 3.75)
the optimal LP solution
Z=41.25 S2: x2 4
the LP optimum value
(1.8, 4)
Z=41
The optimal solution for Subproblem 1 is integral: (3, 3).
If further branching on a subproblem will yield no useful
information, then we can fathom (dismiss) the subproblem.
In our case, we can fathom Subproblem 1 because its solution is integral.
The best integer solution found so far is stored as incumbent.
The value of the incumbent is denoted by Z*.
In our case, the first incumbent is (3, 3), and Z*=39.
Z* is a lower bound for OPT(IP): OPT(IP) Z* .
In our case, OPT(IP) 39. The upper bound is 41: OPT(IP) 41.
Next branching step (graphically)
- Fathom Subproblem 1.
5 - Branch Subproblem 2 on x1 :
(1, 4.44)
Subpr. 4
Subpr. 3 Subproblem 3: New restriction is x1 1.
4 Opt. solution (1, 4.44) with value 40.55
Subproblem 4: New restriction is x1 2.
The subproblem is infeasible
3
Z=40.55
1
Z=20
0 1 2 3 4 5 6 7 8
Solution tree (cont.)
S1: x2 3
int.
All (3, 3) S3: x1 1
(2.25, 3.75) Z=39
(1, 4.44)
Z=41.25
S2: x2 4 Z=40.55
(1.8, 4)
Z=41 S4: x1 2
infeasible
1
Z=20
0 1 2 3 4 5 6 7 8
Solution tree (final)
S1: x2 3 S5: x2 4
int.
All (3, 3) S3: x1 1 (1, 4) int.
(2.25, 3.75) Z=39 Z=37
(1, 4.44)
Z=41.25
S2: x2 4 Z=40.55 S6: x2 5
(1.8, 4) (0, 5)
int.
Z=41 S4: x1 2 Z=40
infeasible
If the optimal value of a subproblem is Z*, then it is fathomed.
In our case, Subproblem 5 is fathomed because 37 39 = Z*.
If a subproblem has integral optimal solution x*,
and its value > Z*, then x* replaces the current incumbent.
In our case, Subproblem 5 has integral optimal solution, and its value
40>39=Z*. Thus, (0,5) is the new incumbent, and new Z*=40.
If there are no unfathomed subproblems left, then the current
incumbent is an optimal solution for (IP).
In our case, (0, 5) is an optimal solution with optimal value 40.
y objective
optimum of
LP relaxation
IP optimum
feasible rounding down optimum
solutions = of LP relaxation
x
Integer Program
Solving MIPs: Branch and Bound
Consider the following integer program:
Maximize x + y + 2z
Subject to 7x + 2y + 3z 36 (IP0)
5x + 4y + 7z 42
2x + 3y + 5z 28
x, y, z 0, integer
Branch & Bound: Example
3
x 1
11
obj 11
5 IP0
y 0 11
z 5 1
11
x 1 x2
Best IP
value
/ 10/ 11
1
0 obj 11.4 obj 10 2 1
5.2 IP2 IP1 2 0 0
0 Best IP
4 4 5
solution
z 5 z6 Integral
1
1 1
obj 11 IP4 IP3
3 3
5
y0 y 1 Infeasible
Integral z4 z 5 z 5
LP constraints
Nutritional requirements
LP constraints
Minimize Cost
0.3x1 + 2.40x2 + 1.30x3 + 0.90x4 + 2.0x5 + 1.9x6
Nutritional requirements
Solution
Weak Duality:
Cost of any feas. diet Price of any feas. pills
MORAL:
The values of the dual variables, which are
always available as part of LP solution, equal
the marginal value of corresponding
resources.
Summary
Summary of branch-and-bound for integer programs
Which variable to branch on?
Which open sub problem to solve first?
Updating the bounds (lower and upper) on OPT(IP)
Summary of fathoming criteria
Optimality test
Branch-and-bound applied to
binary integer programs
mixed integer programs
Mixed integer 0-1 programs
Types of integer programs
All integer programs have linear equalities
and inequalities and some or all of the
variables are required to be integer.
If all variables are required to be integer, then it
is usually called a pure integer program.
If all variables are required to be 0 or 1, it is
called a binary integer program, or a 0-1 integer
program.
if some variables can be fractional and others
are required to be integers, it is called a mixed
linear integer program (MILP) 36
VARIOUS CASES
IN INTEGER
PROGRAMMING
Example
Stockco is considering 6 investments. The
cash required from each investment as well as
the NPV of the investment is given next. The
cash available for the investments is $14,000.
Stockco wants to maximize its NPV. What is
the optimal strategy?
Investment 1 2 3 4 5 6
Cash
Required $5 $7 $4 $3 $4 $6
(1000s)
NPV
added $16 $22 $12 $8 $11 $19
(1000s)
39
Integer Programming
Formulation
What are the decision variables?
1, if we invest in i 1,...,6,
xi
0, else
41
How to model logical constraints
42
Formulating Constraints
Exactly 3 stocks are selected
43
If stock 2 is selected then so is stock 1
The integer programming
constraint:
A 2-dimensional
representation
x1 x2
Stock 2
x1 + x3 1
Stock 3
Stock 1
45
Either stock 4 is selected or stock
5 is selected, but not both.
The integer programming
A 2-dimensional constraint:
representation
x4 + x5 = 1
stock 5
stock 4
46
Representing Non-linear functions
Suppose that the cost of computers is as
follows:
$2,000 each if you buy 1 to 10
$1,800 for each computer from 11 to 25 s.t. 0 x1 10
$1,700 for each computer from 26 to 50 0 x2 15
No more than 50 may be purchased.
0 x3 25
Let x = number of computers bought
x1 = number bought at $2,000 If x1 < 10,
x2 = number bought at $1,800 then x2 = 0.
x3 = number bought at $1,700 If x2 < 15,
then x3 = 0.
Minimize 2,000 x1 +1,800 x2 + 1,700 x3
47
Representing Non-linear functions
w1, w2 {0, 1}
5 6 2 1, 2, 3, 5, 6
7
3 2, 3, 6, 7
4 8 9
11 12 13
10
14 15 16 16 13, 15, 16
51
Representation as Graph Cover
Problem
Replace each district with
1 2 3 a node.
11 12 13
10
14 15 16 x13 + x15 + x16 1
Variables:
let yj = 1 if warehouse j is opened
Let xij = amount of demand for customer i satisfied
via warehouse j. 54
Suppose you knew which warehouses were
open. S = set of open warehouses
xij = demand satisfied for customer
i at warehouse j minimize Si,j cijxij
yj = 1 for j in S, SjS fj
yj = 0 for j not in S.
subject to:
customers get their demand Sj xij = di
satisfied
no shipments are made from an
empty warehouse
xij di if yj = 1
xij = 0 if yj = 0
and x 0
55
More on warehouse location
yj = 1 if warehouse i is
opened
yj = 0 otherwise
minimize Si,j cijxij
xij = flow from i to j
Sj fi yi
subject to:
customers get their demand
satisfied
Si xij = dj
each warehouse is either 0 yj 1
opened or it is not (no partial yj integral for all j.
openings)
xij dj yi for all i, j
no shipments are made from
an empty warehouse and x 0
56
Two key aspects of using integrality
in the model
Costs: we include the cost of the
warehouse only if it is opened.
Si fi yi
57
More on warehouse location
The above is a core subproblem occurring in
supply chain management, and it can be enriched
more complex distribution system
capacity constraints
non-linear transportation costs
delivery times
multiple products
business rules
and more
58
Using Excel Solver to Solve
Integer Programs
Add the integrality constraints (or add that a variable
is binary)
Set the Solver Tolerance. (The tolerance is the
percentage deviation from optimality allowed by
solver in solving Integer Programs.)
The default is 5%
The default is way to high
It often finds the optimum for small
problems
59
Some Comments on IP models
There are often multiple ways of modeling
the same integer program.
60
Example
constraint A: 2x1 + 2x2 + + 2x50 51
constraint B: x1 + x2 + + x50 25
assume that x is binary
constraints C: x1 y, x2 y, , x50 y
(where y is binary)
constraint D: x1 + + x50 50 y
B dominates A, C dominates D
It is not obvious why, until you see the algorithms.
61
Summary on Integer Programming
Dramatically improves the modeling capability
Economic indivisibilities
Logical constraints
Modeling nonlinearities
classical problems in capital budgeting and in
supply chain management
Not as easy to model
Not as easy to solve.
62
Integer Programming
Review
Branch and
Bound 66
Overview of Techniques for
Solving Integer Programs
Enumeration Techniques
Complete Enumeration
list all solutions and choose the best
Branch and Bound
Implicitly search all solutions, but cleverly eliminate the vast majority
before they are even searched
Implicit Enumeration
Branch and Bound applied to binary variables
Cutting Plane Techniques
Use LP to solve integer programs by adding constraints to eliminate the
fractional solutions.
67
Overview of this lecture
Explain and illustrate branch and bound.
It is the starting point for all solution techniques for integer
programming
Lots of research has been carried out over the past 40 years to
make it more and more efficient
But, it is an art form to make it efficient. (We shall get a sense
why.)
Integer programming is intrinsically difficult.
68
Capital Budgeting Example
Investment budget = $14,000
Investment 1 2 3 4 5 6
Cash
Required $5 $7 $4 $3 $4 $6
(1000s)
NPV
added $16 $22 $12 $8 $11 $19
(1000s)
70
An Enumeration Tree
Original
problem
x1 = 0 x1 = 1
x2 = 0 x2 = 1 x2 = 0 x2 = 1
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
71
On complete enumeration
Suppose that we could evaluate 1 billion
solutions per second.
Let n = number of binary variables
Solutions times
n = 30, 1 second
n = 40, 17 minutes
n = 50 11.6 days
n = 60 31 years
72
On complete enumeration
Suppose that we could evaluate 1 trillion
solutions per second, and instantaneously
eliminate 99.9999999% of all solutions as
not worth considering
Let n = number of binary variables
Solutions times
n = 70, 1 second
n = 80, 17 minutes
n = 90 11.6 days
73
n = 100 31 years
Branch and Bound
The essential idea: search the
enumeration tree, but at each node
1. Solve the linear program at the node
2. Eliminate the subtree (fathom it) if
1. The solution is integer (there is no need to go
further) or
2. The best solution in the subtree cannot be as
good as the best available solution (the
incumbent) or
3. There is no feasible solution
74
Branch and Bound
Node 1 is the
44 3/7 original LP
1
Relaxation
Solution at node 1:
x1 =1 x2 = 3/7 x3 = x4 = x5 = 0 x6 =1 z = 44 3/7
Solution at node 2:
x1 = 0 x2 = 1 x3 = 1/4 x4 = x5 = 0 x6 = 1 z = 44 76
Branch and Bound
Node 3 is the
44 3/7 original LP
1
Relaxation plus
x1 = 0 x1 = 1 the constraint
x1 = 1.
44 2 3 44 3/7
44 2 3 44 3/7
x2 = 0
42 44
Solution at node 4: 0 0 1 0 1 1 z = 42
Our first incumbent solution! No solution in
No further searching from node 4 because the subtree can
have a value
there cannot be a better integer solution. 78
better than 42.
Branch and Bound
The incumbent The
is the best incumbent
44 3/7
solution on 1 solution has
hand. value 42
x1 = 0 x1 = 1
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 44 44 5 44 6 44 1/3 7
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
42 4 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43.75 42.66
83
Branch and Bound
1 44 3/7 The
incumbent
x1 = 0 x1 = 1 solution has
value 42
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 4 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43.75 42.66
84
Branch and Bound
1 44 The
incumbent
x1 = 0 x1 = 1 solution has
value 42
44 2 3 44
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43 8 43 9 43 10 43 11 44 12 - 13
43 42 43 42 43 43
42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43 88 43 99 10
43 10 11
43 11 44 12 - 13
43 42 43 42 43 43
42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43 88 43 99 10
43 10 11
43 11 44 12 - 13
42 4 44 5 44 6 44 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
43 88 43 99 10
43 10 11
43 11 44 12 - 13
44 14 15 -
44 16 17 -
89
38 18 19 -
Lessons Learned
Branch and Bound can speed up the search
Only 25 nodes (linear programs) were evaluated
Other nodes were fathomed
Obtaining a good incumbent earlier can be valuable
only 19 nodes would have been evaluated.
Solve linear programs faster, because we start with an
excellent or optimal solution
uses a technique called the dual simplex method
Obtaining better bounds can be valuable.
We sometimes use properties that are obvious to us, such as
the fact that integer solutions have integer solution values
90
Branch and Bound
Notation:
z* = optimal integer solution value
Subdivision: a node of the B&B Tree
Incumbent: the best solution on hand
zI: value of the incumbent
zLP: value of the LP relaxation of the current node
Children of a node: the two problems created for a node, e.g., by saying
xj = 1 or xj = 0.
LIST: the collection of active (not fathomed) nodes, with no active
children.
NOTE: zI z*
91
Illustrating the definitions
The incumbent
44 3/7 solution has
1 value 42
x1 = 0 x1 = 1
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
44 2 3 44 3/7
x2 = 0 x2 = 1 x2 = 0 x2 = 1
42 44 44 5 44 6 44 1/3 7
x3 = 0 x3 = 1 x3 = 0 x3 = 1 x3 = 0 x3 = 1
CASE 3. zI < zLP and xLP is integral. e.g., node 4 was selected,
That is, the LP solution is integral and and the solution to the LP
dominates the incumbent. was integer-valued.
98
Branching
One does not have to have the B&B
tree be symmetric, and one does not
select subtrees by considering variables
in order.
X1 =0 X1 =1 Choosing how to
branch so as to
X2 =0 X2 =1 X8 = 0 X8 = 1 reduce running
time is largely
X =0
art and based
3
3= 0
X
X3 = 0
X3= 1 on experience.
X 3= X 3= 1 X3 =1
X3 = 1 0
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
99
Different Branching Rules are
Possible
Branching: determining children for a node. There are many choices.
100
Different Bounding Techniques are Possible
101
What if the variables are general
integer variables?
One can choose children as follows:
child 1: x1 3 (or xj k)
child 2 x1 4 (or xj k+1)
103