34

MATH 22B NOTES 2012 INTERMEDIATE CALCULUS

3.4. Lagrange multipliers. In this section we nd extrema of a function f on the level set L0 of a function g : Rn R. For example, nd the maximum value of f (x, y ) = x2 xy + y 2 on the unit circle x2 + y 2 1 = 0. The second equation is called the constraint and it should be written as g = 0. So g (x, y ) = x2 + y 2 1. Theorem 3.4.1. If x0 is an extremum of a function f on the level set S given by g = 0 and if g (x0 ) = 0 then there is a real number so that f (x0 ) = g (x0 ) is called the Lagrange multiplier. Proof. When g = 0, it is perpendicular to the level surface S and all vectors perpendicular to the surface at the point x0 must be scalar multiples of g (x0 ). So, if f (x0 ) is not a scalar multiple of g (x0 ) then it is not perpendicular to S and therefore, there is some direction parallel to S in which the function f is increasing. So, the function f cannot have a minimum or maximum or any kind of extremum at the point x0 . Example 3.4.2. Continuing with the example, we must solve the equation f (x0 ) = g (x0 ). Since f = (2x y, x + 2y ) and g = (2x, 2y ), this gives two equations: 2x y = 2x 2y x = 2y Including the constraint, we get three equations in three unknowns. If we add these two we get: x + y = 2(x + y ) 1 So, either = 2 or x + y = 0. Suppose that = 1 . Then the two equations become: 2 2x y = x x = y 2y x = y x = y So, either x = y or x = y . There are exactly 4 points on the unit circle satisfying these conditions: 4 critical pts f type of critical pt 1 1 , 1 1+1=1 min 2 2 2 2 2 2 1 1 1 , +1 +1 =3 max 2 2 2 2 2 2 1 1 , 12 +1 +1 =3 max 2 2 2 2 2 1 1 1 , 1 +1 =1 min 2 2 2 2 2 2

Example 3.4.3. Another example that we did was #27 from the book: Find the extrema of f (x, y, z ) = xy + yz on the surface g (x, y, z ) = xz 1 = 0

MATH 22B NOTES 2012

INTERMEDIATE CALCULUS

35

The gradients are: f = (y, x + z, y ), g = (z, 0, x). So, the Lagrange multiplier equation f = g gives the equations: y = z x + z = 0 = 0 y = x But, the second equation gives x = z . So, x, z have opposite sign. But this is not possible since xz = 1. The conclusion is that the function f has no critical points on the level surface given by g = 0. The dierence between these two examples is that, in the rst example, the level set (the circle) was closed and bounded. Therefore, we know from the lion in the desert argument that the function f must attain a maximum and minimum somewhere on the set. In the second example, the set given by xz = 1 is an hyperboloid. This is closed but unbounded (goes to innity). So, it is OK for the function f not to have critical points on this surface. 3.4.1. Bordered Hessian. There is a second derivative test for Lagrange multiplier using the bordered Hessian. The setup is the following. Given two functions f, g : Rn R and the problem of maximizing or minimizing f on the level set g = 0, we take the function h : Rn+1 R given by h(x, ) = f (x) g (x) In the rst example, n = 2. So, h : R3 R is given by
f

(In the book, is the rst variable.) You take the second derivative of h: 2 2 2 1 2x D f g 2 2 2 2y = H = D h = 1 Dg 0 2x 2y 0 This is called the bordered Hessian. Denition 3.4.4. The signature of a symmetric matrix A is dened to be (a, b, c) where a is the number of positive eigenvalues of A, b is the number of negative eigenvalues of A and c is the nullity of A, the dimension of the null space of A, also equal to the multiplicity of 0 as an eigenvalue of A. So, det A = 0 i c = 0. When c = 0 we usually leave it out. So, the signature is (n, 0) i A is positive denite and the signature is (0, n) i A is negative denite. The crucial point is that det A > 0 i b is even and c = 0. And det A < 0 i b is odd and c = 0. Theorem 3.4.5. The signature of the bordered Hessian D2 h is equal to (1, 1) plus the signature of D2 f on the level set S given by g = 0.

h(x, y, ) = x2 xy + y 2 x2 y 2
g

36

MATH 22B NOTES 2012 INTERMEDIATE CALCULUS

Then det H = 8. Therefore, the signature of H must be (2, 1) and this point is a minimum of f on the level set. 3.4.2. application to quadratic forms. This example f (x, y ) = x2 xy + y 2 is an example of a quadratic form f (x) = xt S x where S is the symmetric matrix 1 1 2 S= 1 1 2 Theorem 3.4.6. If f (x) = xt S x where S is a symmetric n n matrix then f (x) = 2S x. Proof. f (x) = aij xi xj . So, f = akj xj + aik xi xk j i =2 So
j

In the case n = 2, the level set S is one dimensional. So, the signature of D2 F on S is either (1, 0), (0, 1) or (0, 0, 1). These can be distinguished using just the determinant of H: sig of D2 f on S sig of H det H type of crit pt (1, 0) (2, 1) det H < 0 min (0, 1) (1, 2) det H > 0 max (0, 0, 1) (1, 1, 1) det H = 0 unknown 1 1 In the example, the bordered Hessian at the point , with = 1 is 2 2 2 1 1 2 2 2 1 2x 2 2 2y = H = 1 1 1 2 2x 2y 0 2 2 0

akj xj = k th coordinate of S x

f =

f xk

= 2S x

Corollary 3.4.7. Any symmetric matrix S has a real eigenvalue. Note: Any symmetric n n matrix has n real eigenvalues. This can be proved by induction starting with the rst eigenvalue which is given by this theorem. Proof. We want to solve the equation S x = x. So, we take the constraint This is the unit sphere in Rn . This level set is closed and bounded. Therefore any continuous function attains a maximum and minimum on this set. Take the continuous
2 2 g (x) = x2 1 + x2 + + xn 1 = 0

MATH 22B NOTES 2012

INTERMEDIATE CALCULUS

37

function f (x) = xt S x. We know that f attains a maximum at some point x0 on the unit sphere g = 0. Therefore, by the Lagrange multiplier theorem, we know that The gradient of g is easy to calculate: f (x0 ) = g (x0 )

The gradient of f (x) = xt S x is given by the theorem: So, the Lagrange multiplier formula is: Divide by 2 to get: f (x) = 2S x 2S x0 = 2x0

g = (2x1 , 2x2 , , 2xn ) = 2x

S x0 = x0 So, x0 , the point at which f (x) reaches a maximum is an eigenvector of the matrix S ! and the Lagrange multiplier is also the eigenvalue of the symmetric matrix S ! Did Lagrange have this application in mind when he invented Lagrange multipliers? 3.4.3. multiple constrainsts. When there is more than one constraint, say g1 = 0, g2 = 0 then the Lagrange multiplier equation becomes But this can be messy to solve. f = g1 + g2