Review for Midterm II

Math 20
December 4, 2007
Announcements

Midterm I 12/6, Hall A 78:30pm

ML Oce Hours Wednesday 13 (SC 323)

Old exams and solutions on website

Outline
Rank and other Linear Algebra
Linear dependence
Rank
Eigenbusiness
Eigenvector and Eigenvalue
Diagonalization
The Spectral Theorem
Functions of several variables
Graphing/Contour Plots
Partial Derivatives
Dierentiation
The Chain Rule
Implicit Dierentiation
Optimization
Unconstrained Optimization
Constrained Optimization
Rank and other Linear Algebra
Learning Objectives

Determine whether a set of vectors is linearly independent

Find the rank of a matrix

Linear Independence
Denition
Let {a
1
, a
2
, . . . , a
n
} be a set of vectors in R
m
. We say they are
linearly dependent if there exist constants c
1
, c
2
, . . . , c
k
R, not
all zero, such that
c
1
a
1
+ c
2
a
2
+ + c
k
a
n
= 0.
If the equation only holds when all c
1
= c
2
= = c
n
= 0, then
the vectors are said to be linearly independent.
Deciding linear dependence
We showed
a
1
, . . . , a
n
LD c
1
a
1
+ + c
n
a
n
= 0 has a nonzero soln

_
a
1
. . . a
n
_
. .
A
_
_
_
c
1
.
.
.
c
n
_
_
_
. .
c
= 0 has a nonzero soln
system has some free variables
rref(A) has a column with no leading entry to it
Example
Determine if the vectors
_
_
1
0
1
_
_
,
_
_
3
2
2
_
_
,
_
_
0
2
1
_
_
are linearly dependent.
Solution
_
_
_
1 3 0
0 2 2
1 2 1
_
_
_

1
+

_
_
1 3 0
0 2 2
0 1 1
_
_

_
_
_
1 3 0
0 1 1
0 0 0
_
_
_

3
+

_
_
_
1 0 3
0 1 1
0 0 0
_
_
_
So the vectors are linearly dependent.
Example
Determine if the vectors
_
_
1
0
1
_
_
,
_
_
3
2
2
_
_
,
_
_
0
2
1
_
_
are linearly dependent.
Solution
_
_
_
1 3 0
0 2 2
1 2 1
_
_
_

1
+

_
_
1 3 0
0 2 2
0 1 1
_
_

_
_
_
1 3 0
0 1 1
0 0 0
_
_
_

3
+

_
_
_
1 0 3
0 1 1
0 0 0
_
_
_
So the vectors are linearly dependent.
Example
Determine if the vectors
_
_
1
0
1
_
_
,
_
_
3
2
2
_
_
,
_
_
0
2
1
_
_
are linearly dependent.
Solution
_
_
_
1 3 0
0 2 2
1 2 1
_
_
_

1
+

_
_
1 3 0
0 2 2
0 1 1
_
_

_
_
_
1 3 0
0 1 1
0 0 0
_
_
_

3
+

_
_
_
1 0 3
0 1 1
0 0 0
_
_
_
So the vectors are linearly dependent.
Deciding linear independence
So
a
1
, . . . , a
n
LI every column of rref(A) has a leading entry to it
A
_
I
n
O
_
Example
Determine if the vectors
_
_
_
_
1
0
1
1
_
_
_
_
,
_
_
_
_
3
2
2
0
_
_
_
_
,
_
_
_
_
0
2
1
1
_
_
_
_
are linearly dependent.
Solution
_
_
_
_
_
1 3 0
0 2 2
1 2 1
1 0 1
_
_
_
_
_

_
_
_
_
_
1 0 3
0 1 1
0 0 0
1 0 1
_
_
_
_
_

1
+

_
_
_
_
_
1 0 3
0 1 1
0 0 0
0 0 2
_
_
_
_
_

_
_
_
_
_
1 0 0
0 1 0
0 0 1
0 0 0
_
_
_
_
_
So the vectors are linearly independent.
Rank
Denition
The rank of a matrix A, written r (A) is the maximum number of
linearly independent column vectors in A.
If A is a zero matrix, we
say r (A) = 0.
Rank
Denition
The rank of a matrix A, written r (A) is the maximum number of
linearly independent column vectors in A. If A is a zero matrix, we
say r (A) = 0.
Example
Since
rref
_
_
1 3 0
0 2 2
1 2 1
_
_
=
_
_
1 0 3
0 1 1
0 0 0
_
_
this matrix has rank 2.
Example
Since
rref
_
_
_
_
1 3 0
0 2 2
1 2 1
1 0 1
_
_
_
_
=
_
_
_
_
1 0 0
0 1 0
0 0 1
0 0 0
_
_
_
_
this matrix has rank 3.
Another way to compute rank
Theorem
Book Theorem 14.1 The rank of A is the size of the largest
nonvanishing minor of A.
Rank and consistency
Fact
Let A be an mn matrix, b an n 1 vector, and A
b
the matrix A
augmented by b.
Then the system of linear equations Ax = b has a solution (is
consistent) if and only if r (A) = r (A
b
).
Rank and redundancy
Fact
Let A be an mn matrix, b an n 1 vector, and A
b
the matrix A
augmented by b. Suppose that r (A) = r (A
b
) = k < m (m is the
number of equations in the system Ax = b).
Then mk of the equations are redundant; they can be removed
and the system has the same solutions.
Rank and freedom
Fact
Let A be an mn matrix, b an n 1 vector, and A
b
the matrix A
augmented by b. Suppose that r (A) = r (A
b
) = k < n (n is the
number of variables in the system Ax = b).
Then n k of the variables are free; they can be chosen at will
and the rest of the variables depend on them, getting innitely
many solutions.
Outline
Rank and other Linear Algebra
Linear dependence
Rank
Eigenbusiness
Eigenvector and Eigenvalue
Diagonalization
The Spectral Theorem
Functions of several variables
Graphing/Contour Plots
Partial Derivatives
Dierentiation
The Chain Rule
Implicit Dierentiation
Optimization
Unconstrained Optimization
Constrained Optimization
Eigenbusiness
Learning Objectives

Determine if a vector is an eigenvalue of a matrix

Determine if a scalar is an eigenvalue of a matrix

Find all the eigenvalues of a matrix

Find all the eigenvectors of a matrix for a given eigenvalue

Diagonalize a matrix

Know when a matrix is diagonalizable

Eigenbusiness
Denition
Let A be an n n matrix. The number is called an eigenvalue
of A if there exists a nonzero vector x R
n
such that
Ax = x. (1)
Every nonzero vector satisfying (1) is called an eigenvector of A
associated with the eigenvalue .
Example
Midterm II, Fall 2006, Problem 4
Let A =
_
4 2
1 1
_
Problem
Is
_
2
1
_
an eigenvector for A?
Solution
Use the denition of eigenvector:
_
4 2
1 1
__
2
1
_
=
_
6
3
_
= 3
_
2
1
_
So the vector is an eigenvector corresponding to the eigenvalue 3.
Example
Midterm II, Fall 2006, Problem 4
Let A =
_
4 2
1 1
_
Problem
Is
_
2
1
_
an eigenvector for A?
Solution
Use the denition of eigenvector:
_
4 2
1 1
__
2
1
_
=
_
6
3
_
= 3
_
2
1
_
So the vector is an eigenvector corresponding to the eigenvalue 3.
Let A =
_
4 2
1 1
_
Problem
Is 0 an eigenvalue for A?
Solution
The number 0 is an eigenvalue for A if and only if the determinant
of A 0I = A is zero. But
det A = 4 1 1 (2) = 6.
So its not.
Let A =
_
4 2
1 1
_
Problem
Is 0 an eigenvalue for A?
Solution
The number 0 is an eigenvalue for A if and only if the determinant
of A 0I = A is zero. But
det A = 4 1 1 (2) = 6.
So its not.
Methods

To nd the eigenvalues of a matrix A, nd the determinant of

A I. This will be a polynomial in (called the
characteristic polynomial of A, and its roots are the
eigenvalues.

To nd the eigenvector(s) of a matrix corresponding to an

eigenvalue , do Gaussian Elimination on A I.
Diagonalization Procedure

Find the eigenvalues and eigenvectors.

Arrange the eigenvectors in a matrix P and the corresponding

eigenvalues in a diagonal matrix D.

If you have enough eigenvectors so that the matrix P is

square and invertible, the original matrix is diagonalizable and
equal to PDP
1
.
Example
Problem
Let A =
_
2 3
2 1
_
. Diagonalize.
Solution
To nd the eigenvalues, nd the characteristic polynomial and its
roots:
|A I| =

2 3
2 1

= (2 )(1 ) 6
=
2
3 4 = ( + 1)( 4)
So the eigenvalues are 1 and 4.
Example
Problem
Let A =
_
2 3
2 1
_
. Diagonalize.
Solution
To nd the eigenvalues, nd the characteristic polynomial and its
roots:
|A I| =

2 3
2 1

= (2 )(1 ) 6
=
2
3 4 = ( + 1)( 4)
So the eigenvalues are 1 and 4.
To nd an eigenvector corresponding to the eigenvalue 1,
A + I =
_
3 3
2 2
_

_
1 1
0 0
_
So
_
1
1
_
is an eigenvector.
To nd an eigenvector corresponding to the eigenvalue 4,
A 4I =
_
2 3
2 3
_

_
1
3
/2
0 0
_
So
_
3
2
_
is an eigenvector.
Let
P =
_
1 3
1 2
_
so
P
1
=
1
5
_
2 3
1 1
_
Then
A =
_
1 3
1 2
__
1 0
0 4
_
1
5
_
2 3
1 1
_
The Spectral Theorem
Theorem (Baby Spectral Theorem)
Suppose A
nn
has n distinct real eigenvalues. Then A is
diagonalizable.
Theorem (Spectral Theorem for Symmetric Matrices)
Suppose A
nn
is symmetric, that is, A

= A. Then A is
diagonalizable. In fact, the eigenvectors can be chosen to be
pairwise orthogonal with length one, which means that P
1
= P

.
Thus a symmetric matrix can be diagonalized as
A = PDP

,
The Spectral Theorem
Theorem (Baby Spectral Theorem)
Suppose A
nn
has n distinct real eigenvalues. Then A is
diagonalizable.
Theorem (Spectral Theorem for Symmetric Matrices)
Suppose A
nn
is symmetric, that is, A

= A. Then A is
diagonalizable. In fact, the eigenvectors can be chosen to be
pairwise orthogonal with length one, which means that P
1
= P

.
Thus a symmetric matrix can be diagonalized as
A = PDP

,
Outline
Rank and other Linear Algebra
Linear dependence
Rank
Eigenbusiness
Eigenvector and Eigenvalue
Diagonalization
The Spectral Theorem
Functions of several variables
Graphing/Contour Plots
Partial Derivatives
Dierentiation
The Chain Rule
Implicit Dierentiation
Optimization
Unconstrained Optimization
Constrained Optimization
Functions of several variables
Learning Objectives

identify functions, graphs, and contour plots

nd partial derivatives of functions of several variables

Types of functions

linear

polynomial

rational

Cobb-Douglas

etc.
Examples
Problem
In each of the following, nd the domain and range of the function.
Is it linear? polynomial? rational? algebraic? Cobb-Douglas?
(a) f (x, y) = y x
(b) f (x, y) =

y x
(c) f (x, y) = 4x
2
+ 9y
2
(d) f (x, y) = x
2
y
2
(e) f (x, y) = xy
(f) f (x, y) = y/x
2
(g) f (x, y) =
1
_
16 x
2
y
2
(h) f (x, y) =
_
9 x
2
y
2
(i) f (x, y) = ln(x
2
+ y
2
)
(j) f (x, y) = e
(x
2
+y
2
)
(k) f (x, y) = arcsin(y x)
(l) f (x, y) = arctan
_
y
x
_
Graphing/Contour Plots
A function of two variables can be visualized by

its graph: the surface (x, y, f (x, y) in R

3

a contour plot: a collection of level curves

Example
Graph and contour plot of f (x, y) = y x
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = y x
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) =

y x
2
1
0
1
2
2
1
0
1
2
0.0
0.5
1.0
1.5
2.0
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) =

y x
2
1
0
1
2
2
1
0
1
2
0.0
0.5
1.0
1.5
2.0
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = 4x
2
+ 9y
2
2
1
0
1
2
2
1
0
1
2
0
20
40
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = 4x
2
+ 9y
2
2
1
0
1
2
2
1
0
1
2
0
20
40
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = x
2
y
2
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = x
2
y
2
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = xy
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = xy
2
1
0
1
2
2
1
0
1
2
4
2
0
2
4
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) =
y
x
2
2
1
0
1
2
2
1
0
1
2
5
0
5
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) =
y
x
2
2
1
0
1
2
2
1
0
1
2
5
0
5
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) =
1
_
16 x
2
y
2
4
2
0
2
4
4
2
0
2
4
0.0
0.5
1.0
4 2 0 2 4
4
2
0
2
4
Example
Graph and contour plot of f (x, y) =
1
_
16 x
2
y
2
4
2
0
2
4
4
2
0
2
4
0.0
0.5
1.0
4 2 0 2 4
4
2
0
2
4
Example
Graph and contour plot of f (x, y) =
_
9 x
2
y
2
2
0
2
2
0
2
0
1
2
3
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) =
_
9 x
2
y
2
2
0
2
2
0
2
0
1
2
3
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) = ln(x
2
+ y
2
)
2
0
2
2
0
2
1
0
1
2
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) = ln(x
2
+ y
2
)
2
0
2
2
0
2
1
0
1
2
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) = e
(x
2
+y
2
)
2
0
2
2
0
2
0.0
0.5
1.0
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) = e
(x
2
+y
2
)
2
0
2
2
0
2
0.0
0.5
1.0
3 2 1 0 1 2 3
3
2
1
0
1
2
3
Example
Graph and contour plot of f (x, y) = arcsin(y x)
2
1
0
1
2
2
1
0
1
2
1
0
1
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = arcsin(y x)
2
1
0
1
2
2
1
0
1
2
1
0
1
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = arctan
_
y
x
_
2
1
0
1
2
2
1
0
1
2
1
0
1
2 1 0 1 2
2
1
0
1
2
Example
Graph and contour plot of f (x, y) = arctan
_
y
x
_
2
1
0
1
2
2
1
0
1
2
1
0
1
2 1 0 1 2
2
1
0
1
2
The process
To dierentiate a function of several variables with respect to one
of the variables, pretend that the others are constant.
Examples
Example
Let f (x, y) = 3x + 2xy
2
2y
4
. Find both the partial derivatives of
f .
Solution
We have
f
x
= 3 + 2y
2
f
y
= 4xy 8y
3
Examples
Example
Let f (x, y) = 3x + 2xy
2
2y
4
. Find both the partial derivatives of
f .
Solution
We have
f
x
= 3 + 2y
2
f
y
= 4xy 8y
3
Example
Let w = sin cos . Find both the partial derivatives of w.
Solution
We have
w

= cos cos
w

= sin sin
Example
Let w = sin cos . Find both the partial derivatives of w.
Solution
We have
w

= cos cos
w

= sin sin
Example
Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .
Solution
For this its important to remember the chain rule!
f
u
=
1
1 + (u/v)
2

u
u
v
=
1
1 + (u/v)
2
1
v
f
u
=
1
1 + (u/v)
2

v
u
v
=
1
1 + (u/v)
2
u
v
2
Another way to write this is
f
u
=
v
u
2
+ v
2
f
v
=
u
u
2
+ v
2
Example
Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .
Solution
For this its important to remember the chain rule!
f
u
=
1
1 + (u/v)
2

u
u
v
=
1
1 + (u/v)
2
1
v
f
u
=
1
1 + (u/v)
2

v
u
v
=
1
1 + (u/v)
2
u
v
2
Another way to write this is
f
u
=
v
u
2
+ v
2
f
v
=
u
u
2
+ v
2
Example
Let u =
_
x
2
1
+ x
2
2
+ + x
2
n
. Find all the derivatives of u.
Solution
We have a partial derivative for each index i , but luckily theyre
symmetric. So each derivative is represented by:
u
x
i
=
1
2
_
x
2
1
+ x
2
2
+ + x
2
n

x
i
(x
2
1
+ x
2
2
+ + x
2
n
)
=
x
i
_
x
2
1
+ x
2
2
+ + x
2
n
Example
Let u =
_
x
2
1
+ x
2
2
+ + x
2
n
. Find all the derivatives of u.
Solution
We have a partial derivative for each index i , but luckily theyre
symmetric. So each derivative is represented by:
u
x
i
=
1
2
_
x
2
1
+ x
2
2
+ + x
2
n

x
i
(x
2
1
+ x
2
2
+ + x
2
n
)
=
x
i
_
x
2
1
+ x
2
2
+ + x
2
n
Example
Let f (x, y) = 3x + 2xy
2
2y
4
. Find all the second derivatives.
Solution

2
f
x
2
= 0

2
f
x y
= 4y

2
f
y x
= 4y

2
f
y
2
= 24y
2
Example
Let f (x, y) = 3x + 2xy
2
2y
4
. Find all the second derivatives.
Solution

2
f
x
2
= 0

2
f
x y
= 4y

2
f
y x
= 4y

2
f
y
2
= 24y
2
Tangent Planes
Fact
The tangent plane to z = f (x, y) through (x
0
, y
0
, z
0
= f (x
0
, y
0
))
has normal vector (f

1
(x
0
, y
0
), f

2
(x
0
, y
0
), 1) and equation
f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
) (z z
0
) = 0
or
z = f (x
0
, y
0
) + f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
)

This is the best linear approximation to f near (x

0
, y
0
).

is is the rst-degree Taylor polynomial (in two variables) for f .

Tangent Planes
Fact
The tangent plane to z = f (x, y) through (x
0
, y
0
, z
0
= f (x
0
, y
0
))
has normal vector (f

1
(x
0
, y
0
), f

2
(x
0
, y
0
), 1) and equation
f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
) (z z
0
) = 0
or
z = f (x
0
, y
0
) + f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
)

This is the best linear approximation to f near (x

0
, y
0
).

is is the rst-degree Taylor polynomial (in two variables) for f .

Tangent Planes
Fact
The tangent plane to z = f (x, y) through (x
0
, y
0
, z
0
= f (x
0
, y
0
))
has normal vector (f

1
(x
0
, y
0
), f

2
(x
0
, y
0
), 1) and equation
f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
) (z z
0
) = 0
or
z = f (x
0
, y
0
) + f

1
(x
0
, y
0
)(x x
0
) + f

2
(x
0
, y
0
)(y y
0
)

This is the best linear approximation to f near (x

0
, y
0
).

is is the rst-degree Taylor polynomial (in two variables) for f .

Outline
Rank and other Linear Algebra
Linear dependence
Rank
Eigenbusiness
Eigenvector and Eigenvalue
Diagonalization
The Spectral Theorem
Functions of several variables
Graphing/Contour Plots
Partial Derivatives
Dierentiation
The Chain Rule
Implicit Dierentiation
Optimization
Unconstrained Optimization
Constrained Optimization
Fact (The Chain Rule, version I)
When z = F(x, y) with x = f (t) and y = g(t), then
z

(t) = F

1
(f (t), g(t))f

(t) + F

2
(f (t), g(t))g

(t)
or
dz
dt
=
F
x
dx
dt
+
F
y
dy
dt
We can generalize to more variables, too. If F is a function of
x
1
, x
2
, . . . , x
n
, and each x
i
is a function of t, then
dz
dt
=
F
x
1
dx
1
dt
+
F
x
2
dx
2
dt
+ +
F
x
n
dx
n
dt
Fact (The Chain Rule, version I)
When z = F(x, y) with x = f (t) and y = g(t), then
z

(t) = F

1
(f (t), g(t))f

(t) + F

2
(f (t), g(t))g

(t)
or
dz
dt
=
F
x
dx
dt
+
F
y
dy
dt
We can generalize to more variables, too. If F is a function of
x
1
, x
2
, . . . , x
n
, and each x
i
is a function of t, then
dz
dt
=
F
x
1
dx
1
dt
+
F
x
2
dx
2
dt
+ +
F
x
n
dx
n
dt
Tree Diagrams for the Chain Rule
F
x
t
dx
dt
F
x
y
t
dy
dt
F
y
To dierentiate with respect to t, nd all leaves marked t.
Going down each branch, chain (multiply) all the derivatives
together. Then add up the result from each branch.
dz
dt
=
dF
dt
=
F
x
dx
dt
+
F
y
dy
dt
Fact (The Chain Rule, Version II)
When z = F(x, y) with x = f (t, s) and y = g(t, s), then
z
t
=
F
x
x
t
+
F
y
y
t
z
s
=
F
x
x
s
+
F
y
y
s
F
x
t
s
y
t
s
Example
Suppose z = xy
2
, x = t + s and y = t s. Find
z
t
and
z
s
at
(t, z) = (
1
/2, 1) in two ways:
(i) By expressing z directly in terms of t and s before
dierentiating.
(ii) By using the chain rule.
Solution (i)
We have
z = (t + s)(t s)
2
= s
3
ts
2
t
2
s + t
3
So
z
t
= s
2
2ts + 3t
2
z
s
= 3s
2
2ts t
2
Example
Suppose z = xy
2
, x = t + s and y = t s. Find
z
t
and
z
s
at
(t, z) = (
1
/2, 1) in two ways:
(i) By expressing z directly in terms of t and s before
dierentiating.
(ii) By using the chain rule.
Solution (i)
We have
z = (t + s)(t s)
2
= s
3
ts
2
t
2
s + t
3
So
z
t
= s
2
2ts + 3t
2
z
s
= 3s
2
2ts t
2
Solution (ii)
We have
z = xy
2
x = t + s y = t s
So
z
t
=
z
x
x
t
+
z
y
y
t
= y
2
1 + 2xy 1 = (t s)
2
+ 2(t + s)(t s)
z
s
=
z
x
x
s
+
z
y
y
s
= y
2
1 + 2xy(1) = (t s)
2
2(t + s)(t s)
These should be the same as in the previous calculation.
Theorem (The Chain Rule, General Version)
Suppose that u is a dierentiable function of the n variables
x
1
, x
2
, . . . , x
n
, and each x
i
is a dierentiable function of the m
variables t
1
, t
2
, . . . , t
m
. Then u is a function of t
1
, t
2
, . . . , t
m
and
u
t
i
=
u
x
1
x
1
t
i
+
u
x
2
x
2
t
i
+ +
u
x
n
x
n
t
i
In summation notation
u
t
i
=
n

j =1
u
x
j
x
j
t
i
Implicit Dierentiation
The Big Idea
Fact
Along the level curve F(x, y) = c, the slope of the tangent line is
given by
dy
dx
=
_
dy
dx
_
F
=
F/x
F/x
=
F

1
(x, y)
F

2
(x, y)
Tree diagram
F
x
y
x
F
x
+
F
y
_
dy
dx
_
F
= 0
More than two variables
The basic idea is to close your eyes and use the chain rule:
Example
Suppose a surface is given by F(x, y, z) = c. If this denes z as a
function of x and y, nd z

x
and z

y
.
Solution
Setting F(x, y, z) = c and remembering z is implicitly a function
of x and y, we get
F
x
+
F
z
_
z
x
_
F
= 0 =
_
z
x
_
F
=
F

x
F

z
F
y
+
F
z
_
z
y
_
F
= 0 =
_
z
y
_
F
=
F

y
F

z
More than two variables
The basic idea is to close your eyes and use the chain rule:
Example
Suppose a surface is given by F(x, y, z) = c. If this denes z as a
function of x and y, nd z

x
and z

y
.
Solution
Setting F(x, y, z) = c and remembering z is implicitly a function
of x and y, we get
F
x
+
F
z
_
z
x
_
F
= 0 =
_
z
x
_
F
=
F

x
F

z
F
y
+
F
z
_
z
y
_
F
= 0 =
_
z
y
_
F
=
F

y
F

z
Tree diagram
F
x
y
z
x
F
x
+
F
z
_
z
x
_
F
= 0 =
_
z
x
_
F
=
F

x
F

z
Example
Problem 16.8.4
Problem
Let D = f (r , P) denote the deman for an agricultural commodity
when the price is P and r is the producers total advertising
expenditure. Let supply be given by S = g(w, P), where w is an
index for how favorable the weather has been. Assume
g

w
(w, P) > 0. Equilibrium now requires f (r , P) = g(w, P).
Assume that this equation denes P implicitly as a dierentiable
function of r and w. Compute P

w
and comment on its sign.
Solution
We have
f (r , P) g(w, P) 0
f g
r w
P
w
f
P
_
P
w
_
f =g

g
w

g
P
_
P
w
_
f =g
Answer
So
_
P
w
_
f =g
=
g
w
f
P

g
P
f
P
< 0 and
g
P
> 0. We assumed that
g
w
> 0. So in this case,
_
P
w
_
f =g
< 0,
meaning the price decreasing with improving weather.
Outline
Rank and other Linear Algebra
Linear dependence
Rank
Eigenbusiness
Eigenvector and Eigenvalue
Diagonalization
The Spectral Theorem
Functions of several variables
Graphing/Contour Plots
Partial Derivatives
Dierentiation
The Chain Rule
Implicit Dierentiation
Optimization
Unconstrained Optimization
Constrained Optimization
Optimization
Learning Objectives

Find the critical points of a function dened on an open set

(so unconstrained)

Classify the critical points of a function

Find the critical points of a function restricted to a surface

(constrained)
Theorem (Fermats Theorem)
Let f (x, y) be a function of two variables. If f has a local
maximum or minimum at (a, b), and is dierentiable at (a,b), then
f
x
(a, b) = 0
f
y
(a, b) = 0
As in one variable, well call these points critical points.
Theorem (The Second Derivative Test)
Let f (x, y) be a function of two variables, and let (a, b) be a
critical point of f . Then

If

2
f
x
2

2
f
y
2

_

2
f
xy
_
2
> 0 and

2
f
x
2
> 0, the critical point is a
local minimum.

If

2
f
x
2

2
f
y
2

_

2
f
xy
_
2
> 0 and

2
f
x
2
< 0, the critical point is a
local maximum.

If

2
f
x
2

2
f
y
2

_

2
f
xy
_
2
< 0, the critical point is a saddle point.
All derivatives are evaluated at the critical point (a, b).
Example
Problem
Find and classify the critical points of
f (x, y) = 4xy x
4
y
4
Solution
We have
f
x
= 4y 4x
3
and
f
y
= 4x 4y
3
. Both of these are
zero when y = x
3
and x = y
3
So x
9
= x. Since
x
9
x = x(x
8
1) = x(x
4
+ 1)(x
2
+ 1)(x + 1)(x 1)
the real solutions are x = 0, x = 1, and x = 1. The
corresponding y values are 0, 1, and 1. So the critical points are
(0, 0), (1, 1), (1, 1)
Example
Problem
Find and classify the critical points of
f (x, y) = 4xy x
4
y
4
Solution
We have
f
x
= 4y 4x
3
and
f
y
= 4x 4y
3
. Both of these are
zero when y = x
3
and x = y
3
So x
9
= x. Since
x
9
x = x(x
8
1) = x(x
4
+ 1)(x
2
+ 1)(x + 1)(x 1)
the real solutions are x = 0, x = 1, and x = 1. The
corresponding y values are 0, 1, and 1. So the critical points are
(0, 0), (1, 1), (1, 1)
The second derivatives are

2
f
x
2
= 12x
2

2
f
y x
= 4

2
f
x y
= 4

2
f
y x
= 12y
2
So
H(x, y) = 4
_
3x
2
1
1 3y
2
_
At (0, 0), the matrix is
_
0 1
1 0
_
, which has determinant < 0. So
its a saddle point. At the other two points, the matrix is
_
3 1
1 3
_
, which has positive determinant. So those points are
local maxima.
Graph and contour plot of f (x, y) = 4xy x
4
y
4
2
1
0
1
2
2
1
0
1
2
30
20
10
0
2 1 0 1 2
2
1
0
1
2
Theorem (The Method of Lagrange Multipliers)
Let f (x
1
, x
2
, . . . , x
n
) and g(x
1
, x
2
, . . . , x
n
) be functions of several
variables. The critical points of the function f restricted to the set
g = 0 are solutions to the equations:
f
x
i
(x
1
, x
2
, . . . , x
n
) =
g
x
i
(x
1
, x
2
, . . . , x
n
) for each i = 1, . . . , n
g(x
1
, x
2
, . . . , x
n
) = 0.
Note that this is n + 1 equations in the n + 1 variables.
x
1
, . . . , x
n
, .
Problem
Find the critical points and values of
f (x, y) = ax
2
+ 2bxy + cy
2
subject to the constraint that x
2
+ y
2
= 1.
Solution
We have
f

x
= g

x
= 2ax + 2by = (2x)
f

y
= g

y
= 2bx + 2cy = (2y)
So the critical points happen when
_
a b
b c
__
x
y
_
=
_
x
y
_
Problem
Find the critical points and values of
f (x, y) = ax
2
+ 2bxy + cy
2
subject to the constraint that x
2
+ y
2
= 1.
Solution
We have
f

x
= g

x
= 2ax + 2by = (2x)
f

y
= g

y
= 2bx + 2cy = (2y)
So the critical points happen when
_
a b
b c
__
x
y
_
=
_
x
y
_
The critical values are
f (x, y) =
_
x y
_
_
a b
b c
__
x
y
_
=
_
x y
_

_
x
y
_
= (x
2
+ y
2
) =
So

The critical points are eigenvectors!

The critical values are eigenvalues!

The critical values are
f (x, y) =
_
x y
_
_
a b
b c
__
x
y
_
=
_
x y
_

_
x
y
_
= (x
2
+ y
2
) =
So

The critical points are eigenvectors!

The critical values are eigenvalues!