Beruflich Dokumente
Kultur Dokumente
Richard de Neufville
Professor of Engineering Systems
and of
Civil and Environmental Engineering
MIT
Objective
z To develop formally “dynamic programming”,
the method used in lattice valuation of options
— Assumptions: Separability and Monotonicity
— Its optimization procedure: “Implicit Enumeration”
Page 1
Outline
1. Why in this course?
2. Basic concept: Implicit Enumeration
— Motivational Example
3. Key Assumptions
— Independence (Separability) and Monotonicity
4. Mathematics
— Recurrence Formulas
5. Example
6. Types of Problems DP can solve
7. Summary
Page 2
Motivational Example
7 7
7
6 6
6
5 5
5
4 4 4
3 3 3
2 2 2
1 1
1
0
0
0
0 0.5 1 1.5 2 2.5 3 3.5 0 1 2 3 4
0 0.5 1 1.5 2 2.5 3 3.5
I N V E ST M E N T I N VES TM EN T I N V E ST M E N T
7 7
7
6 6
6
5 5
5
4 4 4
3 3 3
2 2 2
1 1
1
0
0
0
0 0.5 1 1.5 2 2.5 3 3.5 0 1 2 3 4
0 0.5 1 1.5 2 2.5 3 3.5
I N V E ST M E N T I N VES TM EN T I N V E ST M E N T
Page 3
Point of Example
z Non-convexity of feasible region can “hide”
optimum solution
z Marginal analysis ,“hill climbing” methods
to search for optimum not appropriate in
these cases (for lattice models in particular)
Semantic Note
z “Dynamic” Programming so named because
— Originally associated with movements through
time and space (e.g., aircraft gaining altitude,
thus “dynamic”)
— “programming” by analogy to “linear
Page 4
Basic Solution Strategy
z Enumeration is basic concept
— This means evaluating “all” the possibilities
— Checking “all” possibilities, we must find best
z HOWEVER…
Engineering Systems Analysis for Design Richard de Neufville ©
Massachusetts Institute of Technology Dynamic Programming Slide 9 of 36
Curse of Dimensionality
z Number of Possible Designs very large
z Example: a simple development of 2 sites,
for 4 sizes of operations over 3 periods
— Number of Combinations in 1 period = 42 = 16
3
— Possibilities over 3 periods = 16 = 4096
Page 5
The Curse -- in lattice model
OUTCOME LATTICE
z End states = N
z Total States ~ Order of only N2 /2
z Number of paths ~ Order of 2N…
— To reach each state at last stage =
1 + 6 +13 +16 + 13 + 6 +1 = 46 paths
Implicit Enumeration
z IE considers all possibilities in principle
z Exploits features of problem to
— Identify classes of dominated possibilities
— Reject these classes
Page 6
Demonstration of I E
200 250
300 150
Page 7
Instead of Costing all Routes… IE
z We find best cost to Omaha (350 via Boise)
z Salt Lake (400), Phoenix (450) routes dominated,
“pruned” – we drop routes with those segments
z Thus don’t look at all Seattle to DC routes
Seattle 100 Boise 500 Fargo Detroit
200 250
300 150
200 250
300 150
Page 8
Some nomenclature definitions
z In Dynamic Programming:
z The Objective Function is G(X)
z Where X = (X1,…. XN ) is the set of states at
each of N “stages” --
z Selection of all Xi defines optimum policy
z giXi are the “return functions” that provide
the effect of an Xi state at the ith stage
z giXi denotes the functional form; Xi the
different states at the ith stage
z So that G(X) = [g1X1,…. gNXN]
Engineering Systems Analysis for Design Richard de Neufville ©
Massachusetts Institute of Technology Dynamic Programming Slide 17 of 36
z “Stages” may
— have a physical meaning, as in example,
— or be conceptual (as the investments in later
Page 9
Useful Concepts -- States
z Each giXi takes on different “states”, that is,
possible situations at a stage
z Examples:
— For cross-country shipment, there are 3 states
(of system, not as “states” of USA) for 1st stage,
Boise, Salt Lake and Phoenix
— For plane accelerating to altitude, a state might
200 250
300 150
Page 10
Solution depends on Decomposition
z Must be able to “decompose” objective
function G(X) into functions of individual
“stages” Xi:
G(X) = [g1X1,…. gNXN]
— Example: cost of Seattle to DC trip can be
Separability
z Objective Function is separable if all giXi are
independent of gJXJ for all J not equal to I
Page 11
Monotonicity
z Objective Function is monotonic if:
improvements in each giXi lead to
improvements in Objective Function, that is if
z given G(X) = [giXi , G’(X’) ] where X = [Xi , X’]
z for all giX’i > giX”i where X’i ,X”i different Xi
● It is true that [giX’i , G’(X) ] > [giX”i , G’(X) ]
Solution Strategy
z Two Steps
— Partial optimization at each stage
— Repetition of process for all stages
option or not
Page 12
Cumulative Return Function
z Result of Optimization at each stage and state
is the “cumulative return function”
Page 13
Application of Recurrence formulas
Page 14
Dynamic Programming Analysis (1)
z At 1st stage the cumulative return function
identically equals return for X1
z That is, f1(X1 ), the best way to allocate
resource over only one stage ≡ g1X1
— There is no other choice
z So f1 (0) = 0
z f1(1) = 2 ; f1 (2) = 4 ; f1 (3) = 6
Page 15
Dynamic Programming Analysis (3)
z LH Column: 0 in no Project f0(0) = 0
z 2nd Column: 0…4 in 1st project, e.g.: f1(2) = 4
A B F
f 0 (0 )= 0 f 1 (0 )= 0 f 2 (0 )= 0
C G
f 1 (1 )= 2 f 2 (1 )= 2
D H
f 1 (2 )= 4 f 2 (2 )= 5
E I M
f 1 (3 )= 6 f 2 (3 )= 7 f 3 (3 )= 8
C G
f 1 (1 )= 2 f 2 (1 )= 2
D H
f 1 (2 )= 4 f 2 (2 )= 5
E I M
f 1 (3 )= 6 f 2 (3 )= 7 f 3 (3 )= 8
Page 16
Contrast DP and Marginal Analysis
z Marginal Analysis:
— reduces calculation burden by only looking at
“best” slopes towards goal, discards others
— Misses opportunities to take losses for later gains
— approach Î 7
z Dynamic Programming :
— Looks at “all” possible positions
— But cuts out combinations that are dominated
Page 17
Formulation Issues
z No standard (“canonical”) form
z Careful formulations required (see text)
z DP assumes discrete states
— thus easily handles integers, discontinuity
— in practice does not handle continuous variables
Page 18