Beruflich Dokumente
Kultur Dokumente
(t), u
(t); u; (t); t) H (x
(t); u
(t); (t); t) ;
where the Hamiltonian function H is dened by
H (x; u; ; t) =
0
F(x; u; t) +
n
X
i=1
i
Q
i
(x; u; t):
Except at points of discontinuity of u
t
,
@H()
@x
i
=
_
i
(t); i = 1; : : : ; n:
Furthermore,
0
= 0 or
0
= 1 and, nally, the following transversality conditions are
satised:
i
(T) 0;
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
3. The Present Value Hamiltonian
The previous theorem suggests that one can solve problem P by simply setting up the
following present value Hamiltonian:
H (x; u; ; t) = F(x; u; t) +
n
X
i=1
i
Q
i
(x; u; t):
2
In the above, we call
i
(t) co-state variables. They are analogous to Lagrange multipliers.
The necessary conditions for a maximum are:
@H()
@u
k
= 0; k = 1; : : : ; m;
@H()
@x
i
=
_
i
(t); i = 1; : : : ; n;
i
(T) 0;
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
The sucient conditions for a maximum are that the Hamiltonian function be concave in
x and u.
Finally, in some cases, our optimization problem will also include other constraints:
max
Z
T
0
F (x(t); u(t); t) dt (P1)
subject to
_ x
i
= Q
i
(x(t); u(t); t); i = 1; : : : ; n;
G
j
(x(t); u(t); t) 0; j = 1; : : : ; q;
x
i
(0) = x
i0
; i = 1; : : : ; n;
x
i
(T) 0; i = 1; : : : ; n;
u(t) 2 ; 2 IR
m
:
In that case, we can write problem P1 as a Lagrangian:
L(x; u; ; ; t) = H(x; u; ; t) +
q
X
j=1
j
G
j
(x; u; t)
= F(x; u; t) +
n
X
i=1
i
Q
i
(x; u; t) +
q
X
j=1
j
G
j
(x; u; t);
where = (
1
; : : : ;
q
). In this case, the necessary conditions for a maximum are:
@L()
@u
k
= 0; k = 1; : : : ; m;
@L()
@x
i
=
_
i
(t); i = 1; : : : ; n;
i
(T) 0;
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
j
(t) 0;
j
(t)Q(x(t); u(t); t) = 0; j = 1; : : : ; q:
The sucient conditions are as before.
3
4. An Economic Example
Consider the following problem:
max
Z
T
0
e
t
u(c(t)) dt
subject to
_ a(t) = ra(t) c(t);
a(0) = a
0
;
a(T) a
T
:
The present value Hamiltonian is
H(a; c; ; t) = e
t
u(c) + [ra c] :
The rst-order conditions are
@H
@c
= 0 =) e
t
u
0
(c(t)) (t) = 0;
@H
@a
=
_
(t) =) (t)r =
_
(t);
(T) 0; (T)a(T) = 0:
A simple interpretation of these rst-order conditions is to relate the growth rate of
consumption, g
c
(t), to the elasticity of intertemporal substitution, (c(t)), and the market
and subjective discount rates, r and . To do this, rst note that
_
(t) = r(t) = re
t
u
0
(c(t)):
Then, totally dierentiating the rst rst-order condition yields
e
t
u
0
(c(t)) +e
t
u
00
(c(t)) _ c(t)
_
(t) = 0:
Dene the growth rate of consumption as g
c
= (1=c)dc=dt:
g
c
(t) =
_ c(t)
c(t)
= (r )
u
0
(c(t))
c(t)u
00
(c(t))
;
such that
g
c
(t) = (c(t))(r );
where (c(t)) = u
0
(c(t))=[c(t)u
00
(c(t))] > 0. Thus, consumption grows whenever the
market discount rate is larger than the subjective discount rate. In that case, the consumer
is less impatient than the market.
4
5. The Innite Horizon Problem
In this section we consider an extension of the nite horizon problem to the case of innite
horizon. To simplify the exposition, we will focus our attention to stationary problems.
The assumption of stationarity implies that the main functions of the problem are
time invariant:
F(x(t); u(t); t) = F(x(t); u(t));
G
i
(x(t); u(t); t) = G
i
(x(t); u(t));
Q
j
(x(t); u(t); t) = Q
j
(x(t); u(t)):
In general, to obtain an interior solution to our optimization problem, we require the
objective function to be bounded away from innity. One way to achieve boundedness is
to assume discounting. However, the existence of a discount factor is neither necessary nor
sucient to ensure boundedness. Thus, we dene
F(x(t); u(t)) = e
t
f(x(t); u(t)):
Our general problem becomes:
max
Z
1
0
e
t
f (x(t); u(t)) dt (P2)
subject to
_ x
i
= Q
i
(x(t); u(t)); i = 1; : : : ; n; (1)
G
j
(x(t); u(t)) 0; j = 1; : : : ; q;
x
i
(0) = x
i0
; i = 1; : : : ; n; (2)
The present value Hamiltonian is
H (x; u; ) = e
t
f(x; u) +
n
X
i=1
i
Q
i
(x; u):
The Lagrangian is
L(x; u; ; ) = e
t
f(x; u) +
n
X
i=1
i
Q
i
(x; u) +
q
X
j=1
j
G
j
(x; u):
The rst-order conditions are:
@L()
@u
k
= 0; k = 1; : : : ; m;
@L()
@x
i
=
_
i
(t); i = 1; : : : ; n;
lim
T!1
i
(T) 0; lim
T!1
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
j
(t) 0;
j
(t)Q(x(t); u(t)) = 0; j = 1; : : : ; q:
5
The sucient conditions are as before.
6. The Current Value Hamiltonian
Any discounted optimal control problem can be solved using both a present value Hamil-
tonian or a current value Hamiltonian. We sometime prefer the current value Hamiltonian
for its simplicity.
We obtain the current value Hamiltonian as follows:
H(x; u; ) = e
t
H(x; u; );
where
i
= e
t
i
. The current value Hamiltonian is thus
H(x; u; ) = f(x; u) +
n
X
i=1
i
Q
i
(x; u):
The transformation from present value to current value aects the rst-order condi-
tions. To see this, let us restate our problem P2 using the current value Hamiltonian. The
problem is
max
Z
1
0
e
t
f (x(t); u(t)) dt (P2)
subject to
_ x
i
= Q
i
(x(t); u(t)); i = 1; : : : ; n; (1)
G
j
(x; u) 0; j = 1; : : : ; q;
x
i
(0) = x
i0
; i = 1; : : : ; n; (2)
The current value Hamiltonian is
H(x; u; ) = f(x; u) +
n
X
i=1
i
Q
i
(x; u):
The Lagrangian is
L(x; u; ; ) = f(x; u) +
n
X
i=1
i
Q
i
(x; u) +
q
X
j=1
j
G
j
(x; u):
6
The rst-order conditions are:
@L()
@u
k
= 0; k = 1; : : : ; m;
@L()
@x
i
=
h
_
i
(t)
i
(t)
i
; i = 1; : : : ; n;
lim
T!1
e
T
i
(T) 0; lim
T!1
e
T
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
j
(t) 0;
j
(t)Q(x(t); u(t)) = 0; j = 1; : : : ; q:
The sucient conditions are as before.
7. The Economic Example in Innite Horizon
Consider the following problem:
max
Z
1
0
e
t
u(c(t)) dt
subject to
_ a(t) = ra(t) c(t);
a(0) = a
0
:
Abstracting from the time subscript, the current value Hamiltonian is
H(a; c; ) = u(c) + [ra c] :
Its rst-order conditions are
@H
@c
= 0 =) u
0
(c) = 0;
@H
@a
=
h
_
i
=) r =
h
_
i
;
lim
T!1
e
t
(T) 0; lim
T!1
e
t
(T)a(T) = 0:
Once again, the interpretation of these rst-order conditions relates consumption
growth to the elasticity of intertemporal substitution, the market discount rate, and the
subjective discount rate, r and . To show this, rst note that
_
= (r ) = (r )u
0
(c):
Then, totally dierentiating the rst rst-order condition yields
u
00
(c) _ c =
_
= (r )u
0
(c);
7
such that
g
c
(t) =
_ c(t)
c(t)
= (r )
u
0
(c(t))
c(t)u
00
(c(t))
or
g
c
(t) = (c(t))(r );
where (c(t)) = u
0
(c(t))=[c(t)u
00
(c(t))] > 0. Thus, consumption grows whenever the mar-
ket discount rate is larger than the subjective discount factor. In that case, the consumer
is less impatient than the market.
8
8. Summary
To summarize, here is the simple cookbook. Assume that you face the following problem:
max
Z
1
0
e
t
f (x(t); u(t)) dt
subject to
_ x
i
= Q
i
(x(t); u(t)); i = 1; : : : ; n;
x
i
(0) = x
i0
; i = 1; : : : ; n:
There are two ways to proceed:
1. Present Value Hamiltonian
Step 1 Write the present value Hamiltonian:
H (x; u; ) = e
t
f(x; u) +
n
X
i=1
i
Q
i
(x; u):
Step 2 Find the rst-order conditions:
@H()
@u
k
= 0; k = 1; : : : ; m;
@H()
@x
i
=
_
i
(t); i = 1; : : : ; n;
lim
T!1
i
(T) 0; lim
T!1
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
2. Current Value Hamiltonian
Step 1 Write the current value Hamiltonian:
H(x; u; ) = f(x; u) +
n
X
i=1
i
Q
i
(x; u):
Step 2 Find the rst-order conditions:
@H()
@u
k
= 0; k = 1; : : : ; m;
@H()
@x
i
=
h
_
i
(t)
i
(t)
i
; i = 1; : : : ; n;
lim
T!1
e
T
i
(T) 0; lim
T!1
e
T
i
(T)x
i
(T) = 0; i = 1; : : : ; n:
9
References
Kamien, Morton I. and Nancy L. Schwartz, 1981, Dynamic Optimization: The Calculus
of Variations and Optimal Control in Economics and Management, New York: North
Holland.
King, Ian P., 1986, A Cranker's Guide to Optimal Control Theory, mimeo Queen's Uni-
versity.
10