TMA4180

Solutions to recommended exercises in Chapter 12 of N&W

Exercise 12.4
We consider the minimization problem
min f (x), (1)
x∈Ω

where f and Ω are convex. We first show that local solutions are also global solutions.
Let x0 be any local solution. Then there exists a neighborhood N (x0 ) such that
f (x0 ) ≤ f (x∗ ), x ∈ N (x0 ) ∩ Ω.
This proof is based on contradiction. Suppose x0 is not a global solution. Then for the
global solution, x∗ , f (x∗ ) < f (x0 ). Since Ω is convex, there exists an α ∈ [0.1] such that
αx0 + (1 − α)x∗ ∈ N (x0 ) ∩ Ω.
Further, by convexity of f ,
f (αx0 + (1 − α)x∗ ) ≤ αf (x0 ) + (1 − α)f (x∗ )
< f (x0 ) + (1 − α)f (x0 ) = f (x0 ).
This contradicts that x0 is a minimum in N (x0 ) ∩ Ω, and hence x0 must be a global
minimum.
Next, we prove that the set of all global solutions, S, is convex. Consider any two
distinct points1 x1 , x2 ∈ S and let x = αx1 + (1 − α)x2 for some α ∈ (0, 1). We need to
prove that x ∈ S. By convexity of Ω, x ∈ Ω, and by convexity of f ,
f (αx1 + (1 − α)x2 ) ≤ αf (x1 ) + (1 − α)f (x2 )
= αf (x1 ) + (1 − α)f (x1 ) = f (x1 ),
since f (x1 ) = f (x2 ) (global solutions). The above inequality must be an equality, or else
x1 is not a global minimum. Thus, x ∈ S, and S is convex.

Exercise 12.15
At x∗ = (0, 1)T , c1 and c2 are the active constraints, and hence λ3 = λ4 = 0. The KKT
conditions at x∗ are now
∂L 3
 
∗
= 2 x1 − + λ1 + λ2 = 0,
∂x1 2
∂L
= 4(x∗2 − t)3 + λ1 − λ2 = 0,
∂x2
λ1 , λ2 ≥ 0.
1
Otherwise, if S consists of only one point, then S is trivially convex.

1
Solving for λ1 and λ2 in the two first equations give
λ1 = 1 − λ2 ,
1 − 4t3
λ2 = .
2
Restricting the Lagrange multiplicators to be non-negative
  gives that |t| ≤ 4−1/3 .
Now, set t = 1. The global minimum of f is 32 , 1 . By strictly convexity of f and
Ω, it follows that only the first constraint is active (convince yourself by an illustration).
Hence, λ2 = λ3 = λ4 = 0, and the KKT conditions reduces to
∂L 3
 
= 2 x1 − + λ1 = 0,
∂x1 2
∂L
= 4(x2 − 1)3 + λ1 = 0,
∂x2
1 − x1 − x2 = 0,
λ1 ≥ 0,
whose solution is (x1 , x2 ) = (0.728, 0.272). By strictly convexity of f and Ω, this is a
global solution.

Exercise 12.17
The KKT condition (12.34a) in N&W can be written as
∇f (x∗ ) − λ∗i ∇ci (x∗ ) = 0
X

i∈A(x∗ )

λ∗i ∇ci (x∗ ) = ∇f (x∗ ).

X
(2)
i∈A(x∗ )

By LICQ, {∇ci (x∗ ), i ∈ A(x∗ )} is linearly independent, and hence (2) has a unique
solution for λ∗i , i ∈ A(x∗ ). Further, by (12.34e) in N&W, λi = 0, for i ∈ I \ A(x∗ ).

Exercise 12.18
The KKT conditions are
∂L
= 2(x∗ − 1) − 2(x∗ − 1)λ∗ = 0,
∂x
∂L
= 2(y ∗ − 2) + 5λ∗ = 0,
∂y
(x∗ − 1)2 − 5y ∗ = 0.
The only solution to this system is x∗ = 1, y ∗ = 0 and λ∗ = 45 . Since ∇c1 (x) =
(2(x − 1), −5)T 6= 0, the LICQ is satisfied.
To show that this is a solution, we use the Second-Order Sufficent Conditions (The-

2
orem 12.6 in N&W). From Eq. (12.55) in N&W,
w ∈ C(x∗ , y ∗ , λ∗ ) ⇒ wT ∇c1 (x∗ , y ∗ ) = 0.
Thus, w2 = 0, and for all w = (w1 , 0) with w1 6= 0,
2
wT ∇2 L(x∗ , y ∗ , λ∗ )w = w12 > 0.
5
Hence, (1, 0) is a solution.
Now, if we substitute the constraint into the objective function, we get the uncon-
strained one dimensional minimization problem
2
1 15

min g(y) = 5y + (y − 2)2 = y + + .
2 4
This problem has the solution ỹ = − 21 , but
1 15
 
g − = < 4 = f (x∗ , y ∗ ),
2 4
so the solution to this problem can not yield solutions for the original problem. In fact,
ỹ can’t be on on the parabola 5y = (x − 1)2 .
Why doesn’t the two minimization problems yield the same solutions? By the con-
straint 5y = (x − 1)2 , it is implicitly given that y ≥ 0, but this is lost when we con-
sider the unconstrained problem min g(y). If we restrict y ≥ 0, then the minimum of
 2
g(y) = y + 1
2 + 15
4 is indeed y = 0, because g 0 (y) > 0 for y > 0.

Exercise 12.19
For this exercise, we only give the correct answers, so you should do the computations
yourself and verify that these are correct.

(a) Yes.

(b) Yes (with λ∗1 = 2/3 and λ∗2 = 5/3).

(c) F(x∗ ) = {d ∈ R2 | − 3d1 − d2 ≥ 0, d2 ≥ 0}, C(x∗ , λ∗ ) = {0}.

(d) Yes (trivially) and yes (nothing to check).

Exercise 12.20
A contour plot of f (x) = x1 x2 is plotted in Figure 1 together with the unit circle. Obvi-
1 1 1 1
ously, both (−2− 2 , 2− 2 )T and (−2− 2 , 2− 2 )T are solutions of the minimization problem.
1 1
It can be shown that all points (±2− 2 , ±2− 2 ) satisfy the KKT conditions, but that
1 1 1 1
only (−2− 2 , 2− 2 )T and (−2− 2 , 2− 2 )T satisfy the second order conditions (the two other
points are maximums).

3
Figure 1: Contour plot of f .