Division of Scientific Computing

Department of Information Technology

Uppsala University

Optimization
Written Examination 2012-12-20
Time: 14:00 - 19:00
Allowed Tools: Pocket Calculator, one A4 paper with notes (machine written, font
size minimum 10 pt)
Maximum number of points: 36 (18 points to pass)
All answers must be motivated the get full points

1. Consider the linear program:

max f (x) = 9x1 + 7x2

s.t. − x1 + 3x2 ≤ 6
(1)
3x1 + 2x2 ≤ 21
x1 , x2 ≥ 0.
a) Rewrite the program into standard form. [0.5 pt]

Solution

min g(x) ≡ −f (x) = −9x1 − 7x2

s.t. − x1 + 3x2 + x3 = 6
(2)
3x1 + 2x2 + x4 = 21
x1 , x2 , x3 , x4 ≥ 0.

b) Solve the problem by using the simplex method. Use the slack variables as
initial basic variables and employ Bland’s rule to determine the entering and
leaving variables. Give the solution for the optimizer x∗ and the value of the
objective function of the optimizer f (x∗ ). [4 pt]
Note: State clearly what your basic and non-basic variables are in every step.

Solution

1.) First iteration

xB = (x3 , x4 )T , xN = (x1 , x2 )T
   
1 0 −1 3
B= , N=
0 1 3 2

1
 cB = (0, 0)T , cN = (−9, −7)
b̂ = B −1 b = (6, 21)T
ŷ T = cTB B −1 = (0, 0)
ĉr = cTN − y T N = (−9, −7)T
The basis is not optimal. The entering variable is x1 (smallest index i in
ĉr,i < 0). On the other hand,
  
−1 1 0 −1
Â1 = B A1 = ,
0 1 3

and, then, the leaving variable is x4 , since only Â1,2 > 0.

2.) Second iteration

xB = (x3 , x1 )T , xN = (x4 , x2 )T
     
1 −1 0 3 −1 1 3 1
B= , N= , B =
0 3 1 2 3 0 1
cB = (0, −9)T , cN = (0, −7)T
b̂ = B −1 b = (13, 7)T
ŷ T = cTB B −1 = (0, −3)
ĉr = cTN − y T N = (3, −1)T
The basis is not optimal since not all components of ĉr are positive. The
negative component corresponds to x2 which is the entering variable. On the
other hand,     
−1 1 3 1 3 1 11
Â2 = B A2 = =
3 0 1 2 3 2
and    
b̂i 39 21
min , A2,i ≥ 0 = min , ,
i Â2,i i 11 2
and, thus, the leaving variable is x3 .
3.) Third iteration

xB = (x1 , x2 )T , xN = (x4 , x3 )T
     
−1 3 1 0 −1 1 −2 3
B= , N= , B =
3 2 0 1 11 3 1
cB = (−9, 7)T , cN = (0, 0)T
1
ŷ T = cTB B −1 = (−3, −34)
11

2
 1
ĉr = cTN − y T N =
(3, 34)T
11
The basis is now optimal since ĉr > 0. Stop here. The optimizer is
1
x∗ = b̂ = B −1 b = (51, 39)T ,
11
and the value of the objective function (with the primal problem written in
standard form) is −732/11.
c) Write down the dual of problem in (1) and give the solution for the dual
problem (the optimizer y ∗ and also value of the dual objective). [1.5 pt]

Solution

The explicit expression of the dual problem is not unique (in the same way
that the primal problem can be also expressed in different forms). For exam-
ple, if we directly calculate the dual from the primal expressed as in (1) we
obtain (see p.14 on the slides for Chapter 6. Duality and Sensitivity for an
identical example)

max f (x) = 9x1 + 7x2 min w(y) = 6y1 + 21y2

s.t. − x1 + 3x2 ≤ 6 s.t. − y1 + 3y2 ≥ 9
⇐⇒ (3)
3x1 + 2x2 ≤ 21 3y1 + 2y2 ≥ 7
x1 , x2 ≥ 0. y1 , y2 ≥ 0.

If, on the other hand, we decide to write first problem (1) in canonical form
we have that the primal and the dual problems can be equivalently expressed
as (see p.12 on the slides for Chapter 6. Duality and Sensitivity)
min g(x) ≡ −f (x) = −9x1 − 7x2 max v(y) ≡ −w(y) = −6y1 − 21y2
s.t. x1 − 3x2 ≥ −6 s.t. y1 − 3y2 ≤ −9
⇐⇒ (4)
− 3x1 − 2x2 ≥ −21 − 3y1 − 2y2 ≤ −7
x1 , x2 ≥ 0, y1 , y2 ≥ 0.

Another alternative is expressing the primal in standard form, as in (2). Then,

the dual problem is
max w(y) = 6y1 + 21y2
min g(x) ≡ −f (x) = −9x1 − 7x2
s.t. − y1 + 3y2 ≤ −9
s.t. − x1 + 3x2 + x3 = 6
⇐⇒ 3y1 + 2y2 ≤ −7 (5)
3x1 + 2x2 + x4 = 21
y1 ≤ 0
x1 , x2 , x3 , x4 ≥ 0.
y2 ≤ 0.
In this last formulation, as the primal constraints are equalities, this would
mean that the variables y1 and y2 are unrestricted, but the constraints y1 ≤ 0

3
 and y2 ≤ 0 (coming for the positivity of the primal—slack—variables x3 and
x4 ) are more restrictive. Note also that the dual in (5) is, of course, equivalent
to the dual in (4) simply by doing the change of variables y1 = −y1 and
y2 = −y2 .
From all the previous equivalent formulations, we have to use the last one
(with the primal expressed in standard form), as the theoretical results of
the weak and strong duality theory are based on the primal problem being
written in such a way (see p.21 on the slides for Chapter 6. Duality and
Sensitivity and pp. 179–180 on the book by Griva, Nash and Sofer (2009)).
Then, the optimizer is just the simplex multiplier from the last iteration i.e.
ŷ = y ∗ = (−3, −34)T /11 and the objective value of the dual function, written
in standard form as in (5), is the same −732/11 as for the primal problem,
by the strong duality theorem.

2. The time evolution of a cooling process in a harmonic quantum system can be

described by
n(t) = e−W t (1 − s) + s .
Here W is the effective cooling rate and s the steady state of the system. Based
on the measured values for average quantum number n in the table, the optimal
values for W and s can be found.
t n
0.5 0.5817
2 0.1739
6 0.1005

a) Formulate the problem as a non-linear least squares problem. Write down

the objective function f (x), the gradient ∇f (x), the Jacobian matrix ∇F (x),
and the expression for the Hessian matrix (without putting in the numbers). [2 pt]

Solution

The problem is formulated in the common form for nonlinear least-squares

problems:
3 3
1X 1X 1
min f (x) = fi (x)2 = [n(ti , x) − ni ]2 = F (x)T F (x),
x 2 2 2
i=1 i=1

where x = (W, s)T and F (x) = (f1 (x), f2 (x), f3 (x))T . The residuals fi (x) are
defined as:
fi (x) = e−W ti (1 − s) + s − ni .

4
 The partial derivatives with respect to the parameters are:
∂fi
= −ti e−W ti (1 − s),
∂W
∂fi
= −e−W ti + 1.
∂s
By using the Jacobian
 
∂f1 ∂f2 ∂f3
 ∂W ∂W ∂W 
∇F (x) =  ∂f ∂f2 ∂f3  ,
1
∂s ∂s ∂s
the derivative of the objective function can be expressed in a more compact
way as:
∇f (x) = ∇F (x)F (x)
The resulting Hessian matrix for the problem is, then:
3
X
∇2 f (x) = ∇F (x)∇F (x)T + fi (x)∇2 fi (x).
i=1

b) Perform one iteration with the Gauss-Newton method with a step length
of one. Start with the initial guess W = 1.46, s = 0.09 (use 4 digits in the
calculation). Check if the Armijo condition for a relaxation parameter µ = 0.1
is fulfilled. Give the value of the objective function. [5 pt]

Solution

We calculate all the components first. The vector of the residuals is then:

F x(0) = (−0.05320, −0.03480, −0.01036)T .

The value of the objective function at x(0) is

1
T
f x(0) = F x(0) F x(0) = 0.0020740.


2

The Jacobian evaluated at x(0) is:

 
(0)

−0.2193 −0.09816 −0.0008566
∇F x = .
0.5181 0.9461 0.9998

The gradient evaluated at x(0) is:

∇f x(0) = ∇F x(0) F x(0) = (0.01509, −0.07083)T .

5
 The approximated Hessian, according to the Gauss-Newton algorithm, is
 

T 0.05771 −0.2073
∇2 f x(0) ≈ H x(0) = ∇F x(0) ∇F x(0) =




.
−0.2073 2.163
Now, having all necessary components, we can calculate a quasi-Newton
search direction:

−1
p = −H x(0) ∇f x(0)


h
T i−1
= − ∇F x(0) ∇F x(0) ∇F x(0) F x(0) = (−0.2193, 0.01173)T ,

and obtain a new point x(1) as

x(1) = x(0) + p = (1.46, 0.09)T + (−0.1704, 0.0122)T = (1.241, 0.1017)T ,

with a function value

f x(1) = 1.060 · 10−5 .

Using the Armijo condition means to make relaxed linear prediction for the
change of the function value:

f x(1) ≤ f x(0) + µpT ∇f x(0) = 1.660 · 10−3 .

And we see the condition is fulfilled, and we have a sufficient decrease in the
step.

3. Consider the following problem

min f (x) = 14 x21 + 18 x22 .

a) Write the algorithm for the steepest-descent method without line search. [0.5 pt]
b) Solve the problem using the algorithm in (3a).
i. Perform the first five iterations of the method starting from x0 = (1, 1)T .
Give a 6-digit approximation to the value of the norm of the gradient at
each iteration. Write the results in a table like the one below. [1.5 pt]
k x p k∇f (x) k
0
1
..
.
ii. What can you say about the convergence rate of the iterates? (do not
make numbers here, just an intuitive interpretation). [1 pt]
iii. Plot the iterates on the x1 x2 plane, together with a sketch of some contour
lines of f (x). [1 pt]
c) Perform again the first iterate as in (3b), but using an exact line search.
Discuss the optimum value obtained for α. [1 pt]

6
Solution

a) The algorithm for the steepest descent method without line search is simply
Input Select initial value for the variable x0 ; k = 0
repeat
Step 1 If xk is optimal, stop
Step 2 Determine a search direction

pk = −∇f (xk )

and update the iterate xk+1 = xk + pk ; k = k + 1

until convergence

b) The gradient of the objective function is

1 
x
∇f (x) = 21 1 .
4 x2

i. So, the first iterate is easily computed as

     
1 1/2 1/2
x1 = x0 + p0 = x0 − ∇f (x0 ) = − 1 = 3 .
1 /4 /4

The rest of the iterates are given in the table below

k x p k∇f (x) k
0 (1, 1) (−1/2, −1/4) 0.5590
1 (1/2, 3/4) (−1/4. − 3/16) 0.3125
2 (1/4, 9/16) (−1/8, −9/64) 0.1881
3 (1/8, 27/64) (−1/16, −27/256) 0.1226
4 ( /16, 81/256)
1 (−1/32, −81/1024) 0.0851
5 ( /32, 243/1024)
1 (−1/64, −243/4096) 0.0613
ii. The convergence rate is linear, something typical of the behavior of the
steepest descent method. (The exact calculations bring a convergence
rate r = 1 and a constant C = 34 ).
iii. The plot with the iterates and some contour lines is given below

7
c) We perform now the first iterate but using a exact line search. To obtain the
value of the optimum step length αopt , we use the exact formula

∇f (x0 )T p0
α=− , (6)
pT0 Qp
1 T
where Q is the Hessian matrix of the quadratic function 2 x Qx − cT x. In
this case,  
1/2 0
2
Q ≡ ∇ f (x) = ,
0 1/4
so the computation is straightforward and α
= 20 9 . Should we take this
1 4
whole step, the new point would be at − 9 , 9 (red line on the plot), with
a gradient norm equal to 0.1242, so we will be improving the performance
of the algorithm. However, should we have used a backtracking strategy, we
would have started with an initial value α = 1, which corresponds to the first
step already calculated in (3b).
Note: In general, when the objective function is not quadratic, (6) cannot
be used, and we should solve the following unidimensional unconstrained op-
timization problem to obtain α

min φ (α) = f (x0 + αp0 ) ,

α

with x0 = (1, 1)T and p0 = − 12 , − 14 as previously calculated. In this case,

the expression for φ(α) is simple,

2
2
φ (x) = 14 1 − α2 + 18 1 − α4 ,
20
so the minimum is easily calculated at α = 9 .

8
4. Consider an inequality-constrained problem with two constraints

min f (x)
s.t. g1 (x) = x21 + x22 + x23 − 3 ≥ 0
g2 (x) = 2x1 − 4x2 + x23 + 1 ≥ 0.

a) State the necessary optimality conditions for this kind of problems, without
using the explicit expressions for g1 and g2 . [0.5 pt]
T T
Consider now the points xa = (1, 1, 1) and xb = (3, 2, −1) .
b) Which of them (if any) is feasible? [0.5 pt]
c) Which of them (if any) is regular? [1 pt]
d) Compute a null-space matrix Z (x) (as stated in 4a) for each point. [1.5 pt]
e) What is the range of admissible values for the Lagrange multipliers if we want
the necessary conditions to be fulfilled at xa ? And at xb ? Can degeneracy
happen in any of these cases? [1.5 pt]

Solution

a) The necessary optimality conditions for inequality-constrained problems (KKT

conditions) are stated on p. 506 on the book of Griva, Nash and Sofer (2009).
Theorem. (Necessary Conditions, Inequality Constraints). Let x∗ be a local
minimum point of f subject to the constraints g(x) ≥ 0. Let the columns of
Z(x∗ ) form a basis for the null space of the Jacobian of the active constraints
at x∗ . If x∗ is a regular point for the constraints, then there exists a vector of
Lagrange multipliers λ∗ such that
• ∇x L (x∗ ; λ∗ ) = 0, or equivalently Z (x∗ )T ∇f (x∗ ) = 0,
• λ∗ ≥ 0,
• λ∗T g (x∗ ) = 0, and
• Z(x∗ )T ∇2xx L(x∗ ; λ∗ )Z(x∗ ) is positive semidefinite.
b) xa is feasible, since it satisfies both constraints

g1 (xa ) = 0 (active), g2 (xa ) = 0 (active).

xb is also feasible, since it also satisfies both constraints

g1 (xb ) = 11 > 0 (inactive), g2 (xb ) = 0 (active).

c) To check for regularity, we need the Jacobian matrix only of the active con-
straints. The general expression of the Jacobian is
 
T 2x1 2x2 2x3
∇g (x) = .
2 −4 2x3

9
 At xa both constraints are active, so the Jacobian evaluated at xa is
 
T 2 2 2
∇g (xa ) = .
2 −4 2

The columns of ∇g (xa ) (or the rows of ∇g (xa )T ) are linearly independent so
xa is a regular point.
Regarding xb , only the second constraint is active, so we need only check the
corresponding column of the Jacobian, i.e.

∇g (xb )T = 2 −4 −2 ,

which is a nonzero vector, thus xb is also a regular point.

d) Z is a basis for the null space of the Jacobian of the active constraints at x.
If we use the variable reduction method, matrices B and N for ∇g (xa ) will
be e.g.
 
       −1 N
 −1
2 2 2 2 1 −B
B= , N= ⇒ B −1 = 16 ⇒ Z (xa ) = =  0 .
2 −4 2 1 −1 Z
1

Regarding ∇g (xb ), matrices B, N and Z will be

 
  2 1
−B −1 N
B = 2 , N = −4 −2 ⇒ B −1 = − 12 ⇒ Z (xb ) =




= 1 0 .
Z
0 1

e) As the two constraints are active at xa , the Lagrange multipliers must be

strictly nonnegative (λ1 , λ2 ≥ 0). Degeneracy happens if λ1 and/or λ2 are
equal to zero. For xb only the second constraint is active, so it must hold that
λ1 = 0 and λ2 ≥ 0. Degeneracy arises if λ2 is equal to zero.

5. Solve the problem

min f (x) = x21 + x1 x2 + x22 − 2x2

s.t. x1 + x2 = 2

two ways analytically:

a) With the necessary conditions.
i. What is x∗ ? [0.5 pt]
ii. What is λ∗ ? [0.5 pt]
iii. Is x∗ a strict local minimizer? [0.5 pt]
iv. Is x∗ also a global minimizer? [0.5 pt]

10
 Note: Solving of the problem by simply expressing one variable in terms
of the other from the constraint, plugging it into the objective function and
minimizing this (now unidimensional) function will NOT be evaluated.
b) Either with a logarithmic barrier function or with a quadratic penalty func-
tion. Motivate your choice.
i. What is x (µ) (or x (ρ), depending on your choice)? [0.5 pt]
ii. What is λ (µ) (or λ (ρ), depending on your choice)? [0.5 pt]
iii. What is x∗ ? [0.5 pt]
iv. What is λ∗ ? [0.5 pt]
c) Compute the Hessian matrix B of the logarithmic barrier function for µ =
10−4 (or of the quadratic penalty function for ρ = 104 , depending on your
choice). What is the condition number of B? What is B −1 ? [2 pt]
Help: For a 2 × 2 nonsingular symmetric matrix Q the condition number is
λmax
cond (Q) = ,
λmin
where λmin , λmax are, respectively, the smallest and largest (in moduli) eigenvalues
of Q.

Solution

This is a linear equality constrained problem, with one single constraint. The
gradient and the Hessian of the objective function are
   
2x1 + x2 2 2 1
∇f (x) = , ∇ f (x) = .
x1 + 2x2 − 2 1 2

The constraint matrix is

A= 1 1 ,
so we can choose (from the variable reduction method), a basis matrix for the null
space of the constraint  
−1
Z= .
1
a) If we impose now the first-order optimality conditions, we get
   
∗ T ∗ 2x1 + x2 1
∇f (x ) = A λ ⇒ = λ,
x1 + 2x2 − 2 1

together with the constraint x1 + x2 − 2 = 0. We solve the gradient condition

for x1 and x2 in terms of λ, and plug x1 (λ) and x2 (λ) to obtain the value
λ = 2 which, in turn, yields x1 = 0 and x2 = 2. Hence

11
 i. x∗ = (0, 2)T .
ii. λ∗ = 2.
iii. Yes, x∗ is a strict local minimizer, since the reduced Hessian at the solu-
tion is   
T 2 ∗

2 1 −1
Z ∇ f (x ) Z = −1 1 = 2 > 0,
1 2 1
satisfying therefore the second order sufficient condition.
iv. x∗ is also a global minimizer since the function is convex and the feasible
set is convex.
b) We solve now the problem using a quadratic penalty function, because the
problem has only equality constraints.
m
gi (x)2 ,
X
1
ψ (x) = 2
i=1

so that the original constrained problem is transformed into the unconstrained

problem

min πρ (x) = f (x) + ρψ (x) = x21 + x1 x2 + x22 − 2x2 + 12 ρ (x1 + x2 − 2)2 .

We now impose the first-order optimality condition to πρ (x)

∂πρ (x)
≡ 2x1 + x2 + ρ (x1 + x2 − 2) = 0
∂x1
∂πρ (x)
≡ x1 + 2x2 − 2 + ρ (x1 + x2 − 2) = 0.
∂x2
Subtracting both equations, we get that x1 = x2 − 2, and plugging it back
into any of the constraints we get that
i. The solution of the unconstrained problem, expressed in terms of ρ is
 
−2 4ρ + 4
x (ρ) = (x1 (ρ) , x2 (ρ)) = , .
2ρ + 3 2ρ + 3

ii. The estimate of the Lagrange multiplier for the constraint is

−4 4ρ
λ (ρ) = −ρg (x (ρ)) = −ρ (x1 + x2 − 2) = −ρ = .
2ρ + 3 2ρ + 3

iii. In the limit when ρ → ∞, x∗ = (0, 2) .

iv. In the limit when ρ → ∞, λ∗ = 2.

12
 c) We need to compute now the Hessian matrix of the quadratic penalty function
πρ (x), obtaining
   
2 2+ρ 1+ρ 10002 10001
B ≡ ∇x πρ (x) = −→ .
1 + ρ 2 + ρ ρ=104 10001 10002

The eigenvalues of this matrix are σ = {1, 2ρ + 3} = {1, 20003} , so the con-
dition number is
cond (B) = 2ρ + 3 = 20003.
Finally, the inverse of B is
!
ρ+2 ρ+1
− 2ρ+3
 
−1 2ρ+3 0.500025 −0.499975
B = ρ+1 ρ+2 −→ ,
− 2ρ+3 2ρ+3 ρ=104 −0.499975 0.500025

which is very close to being singular.

The graphical representation of the problem is given below.

6. A cardboard box for packing quantities of small foam balls is to be manufactured

as shown in the figure. The top, bottom, and front faces must be of double weight
(i.e., two pieces of cardboard). We want to find the dimensions of such a box that
maximize the volume for a given amount of cardboard, equal to 72 m2 .

13
 Top

Front

x
3

x
2

x
1

Bottom

a) Write the optimization problem. State clearly all the constraints (if any). [0.5 pt]
b) By geometric reasoning, the problem is guaranteed to have a solution x∗ =
(x∗1 , x∗2 , x∗3 ) such that x∗1 , x∗2 , x∗3 are strictly positive. Does this fact change
your formulation of the optimization problem? Motivate. [0.5 pt]
c) Through a suitable change of variables, it is possible to transform the problem
in (6b) into a linear equality constrained problem. Formulate one such change
of variables and the corresponding optimization problem. State again clearly
all the constraints p(if any), making use of the assumption in (6b).
Hint: minimizing f (x) for f (x) > 0 is equivalent to minimizing f (x). [1 pt]
d) State the first-order necessary conditions for the problem in (6c). [1 pt]
e) Find x∗ = (x∗1 , x∗2 , x∗3 ). [3 pt]
f) Verify the second-order sufficient condition for x∗ . [1 pt]

Solution

a) The problem can be formulated as

max f (x) = x1 x2 x3
s.t. 4x1 x2 + 3x1 x3 + 2x2 x3 = 72 (7)
x1 , x2 , x3 ≥ 0,

because we need to have double weight on the “Front” face (which has an
area equal to x1 x3 ) and on the “Top” and “Bottom” faces, which have an
area of x1 x2 .
b) As we know that the constraints x1 , x2 , x3 ≥ 0 will not be active at the
solution, we can remove, and we can thus express problem (7) as

max f (x) = x1 x2 x3
s.t. 4x1 x2 + 3x1 x3 + 2x2 x3 = 72. (8)

14
 Problem (8) can be equivalently expressed as a minimization problem

min f (x) = −x1 x2 x3

s.t. 4x1 x2 + 3x1 x3 + 2x2 x3 − 72 = 0. (9)

c) The gradient and the Hessian for problem (9) are

   
x2 x3 0 x3 x2
∇f (x) = − x1 x3  , ∇2 f (x) = − x3 0 x1  . (10)
x1 x2 x2 x1 0

The Jacobian matrix of the nonlinear constraint

g (x) ≡ 4x1 x2 + 3x1 x3 + 2x2 x3 − 72 = 0 (11)

is  
4x2 + 3x3
∇g (x) = 4x1 + 2x3  . (12)
3x1 + 2x2
However, we can write problem (9) in a different (and simpler) way, making
use of the following changes of variables

y 1 = x1 x2 , y 2 = x1 x3 , y 3 = x2 x3 . (13)

Of course, the point y ∗ = (y1∗ , y2∗ , y3∗ ) corresponding to the original solution
x∗ = (x∗1 , x∗2 , x∗3 ) will be also guaranteed to be strictly positive at the solution.
Hence, problem (9) now reads
√
min f (y) = − y1 y2 y3
s.t. 4y1 + 3y2 + 2y3 − 72 = 0.

Since the square root is a monotonically increasing function, we can equiv-

alently minimize the radicand y1 y2 y3 , so we are finally facing the following
linear equality constrained optimization problem

min f (y) = −y1 y2 y3

s.t. 4y1 + 3y2 + 2y3 − 72 = 0. (14)

The gradient and the Hessian of the objective function in problem (14) are
   
y2 y3 0 y3 y2
∇f (y) = − y1 y3  , ∇2 f (y) = − y3 0 y1  .
y1 y2 y2 y1 0

We have only one linear equality constraint

g (y) ≡ 4y1 + 3y2 + 2y3 − 72 = 0, (15)

15
 so the constraint matrix is


A= 4 3 2 ,

and a choice for Z ′ (computed through the variable reduction method) would
be  
−3/4 −1/2
Z′ =  1 0 .
0 1

d) We can now move to the first optimality condition

   
y2 y3 4
T ′
∇f (y ) = A λ ⇒ − y1 y3 = 3 λ′ ,
∗    (16)
y1 y2 2

which has to be solved, together with the feasibility requirement (15) in order
to obtain the stationary points of problem (14).
e) If we solve each equation on the gradient condition (16) for λ′ we obtain
y2 y3 ′ y1 y3 ′ y1 y2
λ′ = − , λ =− , λ =− . (17)
4 3 2
If we compare the right-hand sides of the first and second relationships in
(17), this yields
y2 y3 y1 y3
= ⇒ y2 = 43 y1 (18)
4 3
(here we can divide by y3 as the y1 , y2 , y3 are guaranteed to be strictly pos-
itive). Similarly, if we compare the right-hand sides of the first and third
relationships in (17), we get
y2 y3 y1 y2
= ⇒ y3 = 2y1 . (19)
4 2
We can plug now (18) and (19) into the constraint (15), and solve for y1

4y1 + 3 43 y1 + 2 (2y1 ) − 72 = 0 ⇒ 12y1 = 72 ⇒ y1 = 6.

(20)

Replacing y1 into (17), (18) and (19) this yields y2 = 8 and y3 = 12, obtaining
the solution
y ∗ = (6, 8, 12) , λ′∗ = −24.
The solution in terms of the original variables is obtained from the original
relationships (13)
x1 x2 = 6, x1 x3 = 8, x2 x3 = 12,
dividing the second and the third equations (to solve for x2 in terms of x1 →
x2 = 32 x1 ), and the first and the third equations (to solve for x3 in terms of
x1 → x3 = 2x1 ), and plugging the result into the original constraint (11)
√
4x1 32 x1 + 3x1 (2x1 ) + 2 32 x1 (2x1 ) − 72 = 0 ⇒ x1 = 4 = 2,

16
 and, from here, x2 = 3 and x3 = 4, so the solution is

x∗ = (2, 3, 4) .

We have to check now that this point is actually a stationary point of the
original problem (9). To do so, we check the first order optimality condition
for problem (9), and we observe that the gradient ∇f (x∗ ) and the Jacobian
∇g (x∗ ) are parallel at the solution
   
12 24
∇f (x∗ ) = ∇g (x∗ ) λ ⇒ −  8  = 16 λ,
6 12

obtaining a Lagrange multiplier λ = −2.

f) To check if x∗ = (2, 3, 4) is actually a strict local minimizer, we should move
now to the second order sufficient optimality condition for the original prob-
lem (9), which requires the reduced Hessian

Z T ∇2 f (x∗ ) Z

to be positive definite at x∗ . Here, Z is a null space matrix of the Jacobian

∇g (x∗ ), evaluated at the solution, i.e.
   
24 −2/3 −1/2
∇g (x∗ ) = 16 ⇒ Z =  1 0 .
12 0 1

With this, it is straightforward to check that the reduced Hessian

  
  0 −4 −3 −2/3 −1/2  
T 2 ∗ − /3 1 0 
2 16/3 2
Z ∇ f (x ) Z = −4 0 −2  1 0 =
−1/2 0 1 2 3
−3 −2 0 0 1
 
16
16/3 2
is positive definite, since its principal minors are3 > 0 and    = 12 >
2 3
0, so the point x∗ = (2, 3, 4) is a strict local minimizer of problem (9), and,
in consequence, a strict local maximizer of problem (7).

Good Luck!
Javier & Markus

17