Nonlinear Systems and Control — Spring 2016

and constrained input

−1 ≤ u(t) ≤ 1 ∀t ≥ 0.

Now
H(x, u, λ) = 1 + λ1 x2 + λ2 u.
We notice that ∂H
∂u
= λ2 , i.e. ∂H
∂u
does not depend on u and thus the extremum
is to be reached at the boundaries, i.e. u∗ (t) = ±1, ∀t ≥ 0. Using

u∗ = arg min H(u) = arg min λ2 (t)u(t)

u u

we see that
u∗ (t) = −sign(λ2 (t)).
Figure 8.4 depicts the result of deploying this control. Note that the control
law is discontinuous. Further, we observe that the system now needs less
time to reach the origin than in the previous ”Minimal Energy” example.
Of course, this ”success” is obtained at the cost of deploying more actuator
energy.

8.4 Sufficient Conditions for Optimality

The simplest sufficient condition for optimality is of course to establish
uniqueness of the solution that appears solving the first order necessary con-
dition. In applications it is often the case that this is possible by using
application specific knowledge.
A more generic and well understood sufficient condition follows directly
from the standard constrained optimization result and is can be stated as
follows.

Theorem 8.12 (Sufficiency Conditions I) Let us assume that (x∗ , λ∗ , u∗ )

fulfill the conditions in Theorem 8.5. Then, the condition

∂ 2H ∗ ∗ ∗
(x , u , λ ) > 0
∂u2
is sufficient for the optimality of u∗ .

P. Al Hokayem & E. Gallestey 141 of 201

Nonlinear Systems and Control — Spring 2016

Figure 8.4: Trajectories for Minimal Time Case. Arrival time T ≈ 13.

Proof: The results is proved by contradiction (i.e assuming that u∗ might

not be a local optimizer) and using the expression

∂ 2H ∗ ∗ ∗
H(x∗ , u, λ∗ ) ≈ H(x∗ , u∗ , λ∗ ) + (u − u∗ ) (x , u , λ )(u − u∗ )
∂u2
≥ H(x∗ , u∗ , λ∗ )

and is a well known results from constrained optimization.

Further, useful results can be obtained using the concept of convexity.

Definition 8.13 A function f : Rn → R is called convex when it fulfills the

following property: for any given x and x∗ we have

f ((1 − t)x + tx∗ ) ≤ f (x)(1 − t) + tf (x∗ ), t ∈ [0, 1].

P. Al Hokayem & E. Gallestey 142 of 201

Nonlinear Systems and Control — Spring 2016

A smooth convex function f fulfills the following condition


∗ ∂f 
f (x) ≥ f (x ) + (x − x∗ ), ∀x, x∗ ∈ Rn .
∂x x=x∗
Before stating the first result we also note the following fact for the Hamil-
tonian H = H(x, u, λ).
dH ∂H ∂H ∂u
= + .
dx ∂x ∂u ∂x
if u∗ is an optimal control policy candidate, then ∂H ∂u
= 0 and we establish
that
dH ∂H
(x, u∗ , λ) = (x, u∗ , λ).
dx ∂x

Theorem 8.14 (Sufficiency Conditions II) Let u*(t), and the correspond-
ing x∗ (t) and λ∗ (t) satisfy the minimum principle necessary condition for all
t ∈ [0, T ]. Then, u∗ is an optimal control if H(x, u∗ , λ) is convex in x and
φ(x) is convex in x.
Proof: We want to prove that

J(x0 , u) − J(x0 , u∗ ) ≥ 0,

where x satisfies Equation (8.1). Now u∗ = arg min H and the convexity of
H give us

H(x, u, λ) ≥ H(x, u∗ , λ)

∗ ∗ ∂H 
≥ H(x , u , λ) + (x − x∗ )
∂x x∗ ,u∗ ,λ

Or equivalently

L(x, u) + λf (x, u) ≥ L(x∗ , u∗ ) + λf (x∗ , u∗ ) − λ̇(x − x∗ )

Or

L(x, u) − (x∗ , u∗ ) ≥ −λf (x, u) + λf (x∗ , u∗ ) − λ̇(x − x∗ )

= −λ[(ẋ − ẋ∗ ) − λ̇(x − x∗ )
d
= [−λ(x − x∗ )]
dt
P. Al Hokayem & E. Gallestey 143 of 201
Nonlinear Systems and Control — Spring 2016

Integrating, and using both the transversality condition and the fact that all
trajectories have the same initial condition x = x0 , we observe that
 T
[L(x, u) − L(x∗ , u∗ , λ)]dt = −λ(T )(x(T ) − x∗ (T )) + λ(0))(x∗ (0) − x(0))
0 
∂φ 
= − (x − x∗ )
∂x x∗

Then, since φ is convex, we know that

∂φ
φ(x) − φ(x∗ ) ≥ |x=x∗ (x − x∗ )
∂x
This implies
∂φ
− |x=x∗ (x − x∗ ) = −φ(x) + φ(x∗ ) + α, α ≥ 0.
∂x
We now recall that
 T
J[x0 , u(·)] = φ(x(T )) + L(x(t), u(t))dt.
0

Thus, after simple manipulations we obtain

J(x0 , u) − J(x, u∗ ) = α ≥ 0,

which proves the optimality of u∗ .

P. Al Hokayem & E. Gallestey 144 of 201