Multivariable Calculus, Linear Algebra and Differential Equations

MULTIVARIABLE CALCULUS, LINEAR
ALGEBRA AND DIFFERENTIAL EQUATIONS

Eric A. Carlen
Professor of Mathematics
Rutgers University
September, 2013
Contents
1 GEOMETRY, ALGEBRA AND Analysis IN SEVERAL VARIABLES
1.1
1.2
1.3
1.4
Algebra and geometry in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.1
Vector variables and Cartesian coordinates . . . . . . . . . . . . . . . . . . . .
1.1.2
Parameterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.3
The vector space R
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.1.4
Geometry and the dot product . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.1.5
Parallel and orthogonal components . . . . . . . . . . . . . . . . . . . . . . . . 19
1.1.6
Orthonormal subsets of Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.1.7
Householder reflections and orthonormal bases . . . . . . . . . . . . . . . . . . 24
Lines and planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.2.1
The cross product in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.2.2
Equations for planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.2.3
Equations for lines in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.2.4
Distance problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Subspaces of Rn
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
1.3.1
Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
1.3.2
Orthogonal complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2 DESCRIPTION AND PREDICTION OF MOTION

2.1
2.2
61
Functions from R to R and the description of motion . . . . . . . . . . . . . . . . . . 61

2.1.1
Continuity of functions from R to Rn
2.1.2
Differentiability of functions from R to Rn . . . . . . . . . . . . . . . . . . . . . 63
2.1.3
Velocity and acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.1.4
Torsion and the FrenetSeret formulae for a curve in R3 . . . . . . . . . . . . . 72
2.1.5
Curvature and torsion are independent of parameterization. . . . . . . . . . . . 79
2.1.6
Speed and arc length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
. . . . . . . . . . . . . . . . . . . . . . . 62
The prediction of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

2.2.1
Newtons Second Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.2.2
Ballistic motion with friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2.2.3
Motion in a constant magnetic field and the Rotation Equation . . . . . . . . . 87

i
ii
CONTENTS
2.3
2.2.4
Planetary motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
2.2.5
The specification of curves through differential equations . . . . . . . . . . . . . 97
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3 CONTINUOUS FUNCTIONS
3.1
3.2
3.3
101
Continuity in several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.1.1
Functions of several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.1.2
Continuity in several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.1.3
Continuity and limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.1.4
The squeeze principle is several variables . . . . . . . . . . . . . . . . . . . . . . 110
3.1.5
Continuity versus separate continuity
Continuity, compactness and maximizers . . . . . . . . . . . . . . . . . . . . . . . . . . 117

3.2.1
Open and closed sets in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.2.2
Minimizers and maximizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.2.3
Compactness and existence of maximizers . . . . . . . . . . . . . . . . . . . . . 120
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4 DIFFERENTIABLE FUNCTIONS
4.1
4.2
4.3
4.4
4.5
4.6
. . . . . . . . . . . . . . . . . . . . . . . 113
129
Vertical slices and directional derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . 129

4.1.1
Directional derivatives and partial derivatives . . . . . . . . . . . . . . . . . . . 129
4.1.2
The gradient and a chain rule for functions of a vector variable . . . . . . . . . 132
4.1.3
The geometric meaning of the gradient . . . . . . . . . . . . . . . . . . . . . . . 135
4.1.4
Critical points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.1.5
The gradient and tangent planes . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Linear functions from Rn to Rm
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.2.1
The matrix representation of linear functions . . . . . . . . . . . . . . . . . . . 146
4.2.2
Solving the equation Ax = b for m = n . . . . . . . . . . . . . . . . . . . . . . 151
4.2.3
The distance between two matrices . . . . . . . . . . . . . . . . . . . . . . . . . 155
Differentiability of functions from Rn to Rm . . . . . . . . . . . . . . . . . . . . . . . . 159

4.3.1
Differentiability and best linear approximation in several variables . . . . . . . 159
4.3.2
The general chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

4.4.1
Linear approximation and Newtons iterative scheme . . . . . . . . . . . . . . . 166
4.4.2
The Geometry of Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . 168
4.4.3
Starting and stopping the iteration . . . . . . . . . . . . . . . . . . . . . . . . . 170
The structure of linear transformations from Rn to Rm . . . . . . . . . . . . . . . . . . 174

4.5.1
The Fundamental Theorem of Linear Algebra . . . . . . . . . . . . . . . . . . . 174
4.5.2
Linear independence and Gram-Schmidt orthonormalization . . . . . . . . . . . 175
4.5.3
Computing rank and solving linear equations . . . . . . . . . . . . . . . . . . . 179
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
CONTENTS
iii
5 THE IMPLICIT FUNCTION THEOREM AND ITS CONSEQUENCES

5.1
5.2
5.3
189
Horizontal slices and contour curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

5.1.1
Implicit and explicit descriptions of planar curves . . . . . . . . . . . . . . . . . 192
5.1.2
The tangent line to a contour curve . . . . . . . . . . . . . . . . . . . . . . . . 195
5.1.3
When is the contour curve actually a curve? . . . . . . . . . . . . . . . . . . . . 196
5.1.4
Points on a contour curve where the tangent line has a given slope . . . . . . . 198
Constrained Optimization in Two variables . . . . . . . . . . . . . . . . . . . . . . . . 200

5.2.1
Lagranges criterion for optmizers on the boundary . . . . . . . . . . . . . . . . 202
5.2.2
Application of Lagranges Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 204
5.2.3
Optimization for regions with a piecewise smooth boundary . . . . . . . . . . . 212
The Implicit Function Theorem via the Inverse Function Theorem . . . . . . . . . . . 215
5.3.1
Inverting coordinate tranformations . . . . . . . . . . . . . . . . . . . . . . . . 215
5.3.2
From the Inverse Function Theorem to the Implicit Function Theorem . . . . . 217
5.3.3
Proof of the Inverse Function Theorem. . . . . . . . . . . . . . . . . . . . . . . 218
5.4
The general Implicit Function Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.5
Implicit curves and surfaces in three or more dimensions . . . . . . . . . . . . . . . . . 224

5.5.1
5.6
Constrained optimization in three variables . . . . . . . . . . . . . . . . . . . . 225
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
6 CURVATURE AND QUADRATIC APPROXIMATION

6.1
6.2
6.3
The best quadratic approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

6.1.1
Higher order directional derivatives and repeated partial differentiation . . . . 235
6.1.2
A multivariable Taylor expansion . . . . . . . . . . . . . . . . . . . . . . . . . . 238
6.1.3
Clairaults Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6.1.4
Eigenvalues and eigenvectors of a symmetric matrix . . . . . . . . . . . . . . . 244
6.1.5
Principle curvatures at a critical point for functions of 2 variables . . . . . . . . 249
6.1.6
Types of critical points for real valued functions on Rn . . . . . . . . . . . . . . 257
6.1.7
Sylvesters Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Curvature of surfaces in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

6.2.1
Parameterized surfaces in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
6.2.2
Principal curvatures and Gaussian curvature . . . . . . . . . . . . . . . . . . . 264
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
7 INTEGRATION IN SEVERAL VARIABLES

7.1
7.2
235
277
Integration and summation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

7.1.1
A look back at integration in one variable . . . . . . . . . . . . . . . . . . . . . 277
7.1.2
Integrals of functions on R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
7.1.3
Computing area integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
7.1.4
Polar coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Jacobians and changing variables of integration in R2

7.2.1
. . . . . . . . . . . . . . . . . . 293
Letting the boundary of D determine the disintegration strategy . . . . . . . . 293
iv
CONTENTS
7.2.2
7.3
7.4
7.5
The change of variables formula for integrals in R2 . . . . . . . . . . . . . . . . 299
Integration in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
7.3.1
Reduction to iterated integrals in lower dimension . . . . . . . . . . . . . . . . 304
7.3.2
The change of variables formula for integrals in R3 . . . . . . . . . . . . . . . . 306
Integration on parameterized surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

7.4.1
Parameterized surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
7.4.2
The tangent plane to a differentiable parameterized surface . . . . . . . . . . . 312
7.4.3
The surface area of a parameterized surface . . . . . . . . . . . . . . . . . . . . 315
Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
Chapter 1
GEOMETRY, ALGEBRA AND

Analysis IN SEVERAL
VARIABLES
1.1
1.1.1
Algebra and geometry in Rn

Vector variables and Cartesian coordinates
Our subject, multivariable calculus, is concerned with the analysis of functions taking several variables
as input, and returning several variables of output. Many problems in science and engineering
lead to the consideration of such functions. For instance, one such function might give the current
temperature, barometric pressure, and relative humidity at a given point on the earth, as specified
by latitude and longitude. In this case, there are two input variables, and three output variables.
You will also recognize the input variables in this example as coordinates. In fact, in most
examples the variables we consider will arise either as coordinates or parameters of some kind. Our
subject has its real beginning with a fundamental idea of Rene Descartes, for whom Cartesisan
coordinates are named.
Descartes idea was to specify points in three dimensional Euclidean space using lists of three
numbers (x, y, z), now known as vectors. To do this one first fixes a reference system, by specifying
a base point or origin that we shall denote by 0, and also a set of three orthogonal directions.
For instance, if you are standing somewhere on the surface of the Earth, you might take the point at
which you stand as the origin 0, and you might take East to be the first direction, North to be the
second, and straight up to be the third. All of these directions are orthogonal to one another. Let
us use the symbols e1 , e2 and e3 to denote these three directions.
c 2012 by the author.

CHAPTER 1. GEOMETRY, ALGEBRA AND ANALYSIS IN SEVERAL VARIABLES
The brilliant idea of Rene Descartes is this:

We can describe the exact position of any point in physical space by telling how to reach it by moving
in the directions e1 , e2 and e3 . This is simply a matter of giving directions: Start at the origin 0,
and go out x units of distance in the e1 direction, then go out y units of distance in the e2 direction,
and finally go out z units of distance in the e3 direction. The numbers x, y and z may be positive of
negative (or zero). If, say, x is negative, this means that you should go |x| units of distance in the
direction opposite to e1 .
Thus, following Descartes idea, we can specify the exact position of any point in physical space
by giving the ordered list of numbers (x, y, z) that describes how to reach it from the origin of our
reference system. The three numbers x, y and z are called the coordinates of the point with respect
to the given reference system. (It is important to note that the reference system is part of the
description too: Knowing how far to go in each direction is not much use if you do not know the
directions, or the starting point.)
This representation of points in space as ordered triples of numbers, such as (x, y, z) allows one to
use algebra and calculus to solve problems in geometry, and it literally revolutionized mathematics.
1.1. ALGEBRA AND GEOMETRY IN RN
We now define a three dimensional vector to be an ordered triple (x, y, z) of real numbers. The
geometric interpretation is that we regard x, y and z as the coordinates of a unique point in physical
space the one you get to by starting from the origin and moving x units of distance in the e1
direction, y units of distance in the e2 direction, and z units of distance in the e3 direction. We may
identify the vector (x, y, z) with this point in physical space, once again keeping in mind that this
identification depends on the reference system, and that what the vector really represents is not the
point itself, but the translation that carries the origin to that point.
As we have said, this way of identifying three dimensional vectors with points in physical space is
extremely useful because it brings algebra to bear on geometric problems. For instance, referring to
the previous diagram, you see from (two application of) the Pythagorean Theorem that the distance
from the origin 0 to the point represented by the vector (x, y, z) is
p
x2 + y 2 + z 2 .
This distance is called the length or magnitude of the vector x = (x, y, z). The vector x also has
a direction; namely the direction of the displacement that would carry one directly from 0 to x. It is
useful to associate to each direction the vector corresponding to a unit displacement in that direction.
This provides a one-to-pne correspondence between directions and unit vectors, i.e., vectors of unit
length.
The unit sphere is defined to be the set of all unit vcctors; i.e., all points a unit distance from
the origin. Thus, a point represented by the vector x = (x, y, z) lies on the unit sphere if and only if
x2 + y 2 + z 2 = 1 .
(1.1)
You are probably familiar with this as the equation for the unit sphere. But before Descartes,
geometry and algebra were very different subjects, and the idea of describing a geometric object in
terms of an algebraic equation was unknown. It revolutionized mathematics.
1.1.2
Parameterization
Writing down the equation for the unit sphere is only a first step towards solving many problems
involving spheres, such as, for example, computing the surface area of the unit sphere. Often the
second step is to solve the equation. Now, for an equation like x2 = 1, we can specify the set of
all solutions by writing it out: {1, 1}. But for x2 + y 2 + z 2 = 1, there are clearly infinitely many
solutions, and we cannot possibly write them all down.
What we can do, however, is to parameterize the solution set. Let us go through an example
before formalizing this fundamental notion. Better yet, let us start with something even simpler:
Consider the equation
x2 + y 2 = 1
(1.2)
in the x,y plane. (The x,y plane is the set of points (x, y, z) with z = 0.) You recognize (1.2) as the
equation for the unit circle in the x,y plane. Recall the trigonometric identity
cos2 + sin2 = 1 .
(1.3)
Thus for all , the points

(x, y) = ( cos , sin )
solve the equation (1.2).
Conversely, consider any solution (x, y) of (1.2). From the equation, 1 x 1, and hence we
may define
arccos(x) y 0
:=
arccos(x) y < 0
(1.4)
By the definition of the arccos function, < . Since cos is an even function of , it follows
that x = cos , and then one easily sees that y = sin .
Thus, we have a one-to-one correspondence between the points in the interval (.] and the
set of solutions of (1.2). The correspondence is given by the function
7 ( cos , sin )
(1.5)
from (.] onto the unit circle. This is an example of a parameterization: As the parameter varies
over (.], ( cos , sin ) varies over the unit circle, covering each point for exactly one value of the
parameter .
Since the function in (1.4) is one-to-one and onto, it is invertible. The inverse is simply the map
(x, y) 7
(1.6)
where for x and y solving x2 + y 2 = 1, is given by (1.4). The function in (1.6) is called the angular
coordinate function on the unit circle. As you see in this example, finding a parameterization of the
solution set of some equation and finding a system of coordinates on the solutions set are two aspects
of the same thing. We will make formal definitions later; for now let us continue with examples.
For r > 0,
x2 + y 2 = r 2
is the equation of the centered circle of radius r in the x,y plane. Since
x 2 y 2
x2 + y 2 = r 2
+
=1,
r
r
we can easily transform our parameterization of the unit circle into a parameterization of the circle
of radius r: The parameterization is given by
7 (r cos , r sin )
while its inverse, the coordinate function, is given by
p

arccos x/ x2 + y 2

(x, y) 7 :=
arccos x/px2 + y 2
(1.7)
y0
(1.8)
y<0
which specifies the angular coordinate as a function of x and y. Since x2 + y 2 = r2 > 0, we never
divide by zero in this formula.
For our next example, let us parameterize the unit sphere; i.e., the solution set of (1.1). Note
that x2 + y 2 + z 2 = 1 implies that 1 z 1. Recalling (1.3) once more, we define
= arccos(z) ,
(1.9)
so that 0 , and z = cos .

It follows from (1.1) and (1.3) that x2 + y 2 = sin2 , and then, since sin 0 for 0 ,
sin =
x2 + y 2 .
(1.10)
Evidently, for x and y not both zero, (x, y) lies on the circle of radius sin . We already know
how to parameterize this: Setting r = sin in (1.7), the parameterization map is
7 ( sin cos , sin sin ) = (x, y) .
(1.11)
Since (1.9) gives us z = cos , we combine results to obtain the parameterization

(, ) 7 ( sin cos , sin sin , cos ) = (x, y, z) .
Note that is only defined when at least one of x and y is not zero, and so is not defined at
(0, 0, 1), the North pole and (0, 0, 1), the South pole. These points correspond to = 0 and
= respectively. However, apart from these two points, every point on the unit sphere is of the
form ( sin cos , sin sin , cos ), for exactly one pair of values (, ) in the rectangle (, ] (0.)
The function
(, ) 7 ( sin cos , sin , sin , cos ) ,
is therefore a parameterization of the unit sphere, take away the North and South poles.
The inverse function,
(x, y, z) 7 (, )
where is given by (1.7) and is given by (1.9), is a standard coordinate coordinate function on the
sphere. These coordinates are essentially the usual longitude and latutude coordinates, except the
here we measure latitide from the North pole instead of the equator.
Our final example in this subsection opens perspectives on several issues that will concern us in
this course. The problem of the description of rigid body motion, which is fundamentally important
in physics, robotics, aeronautical engineering, computer graphics and other fields, and yet can be
discussed without any background in any of them:
Imagine a solid, rigid object moving in three dimensional space. To keep the picture simple,
suppose the object is a cube shaped box. Here is a picture showing the box shaped object at two
times: t = 0 and t = 1:
Before trying to describe the motion of this object, let us start with a simpler problem:
How can we describe, in precise mathematical terms, the way that the box is situated in three
dimensional physical space at any given time, say t = 0?
First of all, on the box itself, we fix one corner once and for all to be the base-point in the
cube, and we label the three edges meeting at this corner as edge one, edge two and edge three.
Now we can specify the configuration of the box by specifying four vectors: x, u1 , u2 and u3 .
Here is how: The vector x tells us the position of the base-point corner of the cube: x = (x, y, z)
is the vector of Cartesian coordinates of location of the base point.
This is a start, but knowing where the base point is does not provide full knowledge of the
configuration of the box; i.e., of how the box is positioned in physical space keeping the base point
fixed, one could rotate the cube around into infinitely many different configurations. This is where
the three vectors u1 , u2 and u3 come in: The box is to be positioned so that edge number one runs
along the direction of u1 , edge number two runs along the direction of u2 , and edge number three
runs along the direction of u3 , as in the diagram.
Thus, knowing the vectors x, u1 , u2 and u3 provides full knowledge of the configuration of the
box. Notice that in our description of the configuration, both the length and the direction of x are
important, but it is only the directions of u1 , u2 and u3 that matter, which is why we may take them
to be unit vectors.
To describe the motion of the box in physical space, we then need only to give the four vectors
x, u1 , u2 and u3 at each time t. This leads us to consider the four vector valued functions x(t),
u1 (t), u2 (t) and u3 (t). We can specify the situation in space of our box as a function of the time t
by giving four vector valued functions x(t), u1 (t), u2 (t) and u3 (t). Each of these four vectors has
there entries to keep track of, so we would be have a total of 12 coordinates to keep track of. That
is beginning to look complicated.
However, such a description involves quite a lot of redundant information. For example, as we
explain next, essentially all of the information in u3 (t) is redundant, and we can reduce the number
of variables we need to keep track of.
The explanation of why u3 (t) is redundant turns on a very important point about Cartesian
coordinate systems: These all involve a reference frame of three orthonormal vectors, giving the
directions of the three coordinate axes. It turns out that there a two kinds of reference frames: right
handed and left handed, and if you know any two of the vectors in a reference frame, and know
whether it is right handed or left handed, then you also know the third.
Here is the picture that explains the names:
Let u1 , u2 and u3 be the three orthogonal directions. If you can rigidly rotate your right hand
(keeping the index finger, middle finger and thumb orthogonal) around so that your index finger
points in the direction of u1 , your middle finder points in the direction of u2 , and your thumb points
in the direction of u3 , as in the picture, then the frame {u1 , u2 , u3 } is right handed. Otherwise, it is
left handed. We will give a mathematically precise definition later in Chapter One, and connect it
with this picture in Chapter Two.
For now, let us go back to the diagram in this section showing the situation in space of our
cubical box at the times t = 0 and t = 1. We have drawn in the frame {u1 , u2 , u3 } of unit vectors
running along the edges coming out of the base point corner at both times. You should check that
we have labeled them so that in both cases, the frame is right handed. Moreover, and this is the key
point,
As the box moves, carrying the three vectors along, the frame remains right-handed.
Making a precise mathematical statement out of this requires that we make use of the notion
of continuous motion, which we study in Chapter Two. For now though, you will probably find this
statement intuitively reasonable for any sort of physically possible rigid motion.

Now suppose you know the first two orthogonal unit vectors u1 and u2 in the set of orthogonal
directions for a coordinate system. Then there are only two directions that are oathogonal to u1
and u2 : Referring to the picture above, there are the right-handed, thumbs-up direction, and the
left-handed, thumbs-down direction. If we know we are working with a right-handed coordinate
system, u3 must be the right-handed, thumbs-up direction shown in the picture. Knowing u1 and
u2 , align your index finger along u1 and your middle finger along u2 , as in the picture. Your thumb
now points along u3 .
Given that our frame is right handed, all information about the frame is contained in the first and
second vectors; the third vector is determined by these, and the right hand rule.
Thus, all of the information in u3 (t) is redundant. We get a complete description of the configuration of the box in space by giving the list of three vectors x(t), u1 (t) and u2 (t), provided we make
the decision to work with right handed frames.
However, there is still redundancy in this description. As we now explain, we can parameterize
the set of all right handed frames in three dimensional space in terms of three angles.
First, let us parameterize the choices for u1 . The vector u1 corresponds to a point on the unit
sphere in three dimensional space. As we have seen, to each such unit vector u1 (except the North
and South poles), there corresponds a unique latitude (0, ) and longitude (, ]
specifying the point. such that
u1 = ( sin cos , sin sin , cos ) .
(Even the North and South poles are included if we include the values = 0 and = and ignore
the fact that is undefined, since when = 0 or = , sin = 0, and the value of does not
matter.)
Next, when we go on to specify u2 , we do not get two additional angles, but only one: Two
additional angles are what we would get if we could make an arbitrary choice in the unit sphere for
u2 , but we cannot: We must keep u2 orthogonal to u1 .
To see all of the points on the unit sphere that are orthogonal to u1 , look at the great circle
where the plane through the origin that is orthogonal to the line through u1 intersects the sphere.
This intersection is a circle, and all of the unit vectors that are orthogonal to u1 lie on this circle.
Here is a picture shown a sphere that is sliced by the plane through the origin that is orthogonal
to u1 , with the top half pulled away for better visibility:
The edge of the slice is the great circle on which u2 must lie. We can parameterize this circle
by choosing as a reference point the unique highest point on the circle (where we are assuming again
the 6= 0, ). We can then parameterize the circle by an angle [0, 2) that runs counter-clockwise
around the circle when viewed from above, starting at = 0 at our reference point.
Thus, the two angles and determine u1 , and then the third angle determines u2 , and then
the right -hand rule determines u3 . Therefore, there is a one-to-one correspondence between right
handed orthonormal frames {u1 , u2 , u3 } in which u1 is not either straight up or straight down, and
triples (, , ) in the set (0, ) [0, 2) [0, 2). These angles , and are the parameters in
our parameterization of the set of (almost all) right-handed frames. To specify what the frame
{u1 (t), u2 (t), u3 (t)}, we only need to specify the three angles (t), (t) and (t).
Therefore, to specify the frame {u1 , u2 , u3 } attached to the box, we only need to give the three angles
, and . There is no remaining redundancy in this description.
Thus, we can completely describe, without any redundancy, the configuration of our box as a
function of time by giving a list of six numerically valued functions:
(x(t), y(t), z(t), (t), (t), (t))
where the first three entries are the Cartesian coordinates of the base-point, and the second three
entries are the three angles specifying the orientation of the box in physical space.
We have written this list out in the same way we write three dimensional vectors; however, this
list has six entries, and they are of different types: The first three represent signed distances, and
the last three represent angles. In fact, it will turn out to be useful think of this more complicated
object as a vector too - a vector in 6 a dimensional space. Several observation are worthwhile at this
point:
(1) We have just parameterized, with a few details postponed for now, an interesting set of mathematical objects, namely the set of right handed orthonormal frames in three dimensional space. We
will be working with parameterizations of all sorts of mathematical sets in this course, and not only
solutions sets of equations, and though one can often introduce an equation describing the set, it is
not always the key to finding a parameterization of the set.
10
(2) We have just seen how a simple and very natural problem leads to the consideration of functions
with values in a space of higher dimensional vectors; i.e., vectors with more than three entries.
However, that is not all: We will also be concerned with functions whose argument is a such a
vector. For example, if various conservative forces are acting on our box, there will be an energy
E associated to each configuration (x, y, z, , , ) of the box. This yields a function
E(x, y, z, , , )
defined on a set of 6 dimensional vectors with values in the real numbers. A standard problem,
important in physics, mechanical engineering and other fields, is to find the values of x, y, z, ,
and that minimize this function. One subject that we shall study in detail is the use of calculus
for finding minima and maxima in this multi-variable setting.
Geometry is very helpful in finding minima and maxima for functions of several variables, and
Descartes idea is essential here. Let us briefly explain why, since otherwise you may wonder why we
shall spend so much time discussing algebra and geometry at the beginning of this calculus course.
You learned in single variable calculus that the if the tangent line to the graph of y = f (x) is
not horizontal at some point x0 in the interior (a, b) of the interval [a, b], then x0 cannot possibly
minimize or maximize the function f , even locally: Since the graph has a non-zero slope, you can
move to higher or lower values by moving a little bit to either the left or to the right. Hence the
only candidates for interior maxima and minima are the critical points; that is, points at which the
tangent line to the graph is horizontal.
Now consider a very simple real valued function f (x, y) = x2 + y 2 . The graph of this function is
the set of points (x, y, z) for which z = f (x, y); i.e., z = x2 + y 2 . This graph is a parabolic surface
in three dimensional space. At each point on the surface there is a tangent plane, which is the plane
that best fits the graph at the point in a sense quite analogous to the sense in which that tangent
line provides the best fit to the graph of a single variable differentiable function at a given point.
Here is a three a picture showing the portion of the graph of z = x2 + y 2 for 2 x, y 2,
together with the tangent plane to this graph at the point with x = 1 and y = 1.
15
10
5
0
5
10
3
2 1
Here is another picture of the same thing from a different vantage point, giving a better view of
the point of contact:
11
15
10
5
0
5
10
3 2 1
As you can see, the tangent plane is tilted, so there are both uphill and downhill directions at
this point, and so (x, y) = (1, 1) cannot possibly minimize or maximize f (x, y) = x2 + y 2 . Of course,
for such a simple function, there are many ways to see this. However, for more interesting functions,
this sort of reasoning in terms of tangent planes will be very useful.
To make use of this sort of reasoning, we first need effective means of working with lines and
planes and such. For example, every plane has an equation that specifies it, just like x2 + y 2 + z 2 = 1
specifies the unit sphere. What is the equation for the tangent plane pictures above? In this course,
we will learn about a higher dimensional version of the derivative that determines tangent planes in
the same way that the single variable derivative specifies tangent lines. And, as indicated above, we
not only need the two or three dimensional version of this, but a version that works for any number
of dimensions though in more than two variables it will be a tangent hyperplane that we will be
computing.
There are many other subjects we shall study involving the calculus both integral and differential of functions that take vectors as input, or return them as output, or even both.
Multivariable functions are simply functions that take an ordered list of numbers as their input, or
return an ordered list as output, or both.
In the next section, we begin developing the tools to work with them. We use these tools
throughout the course.
1.1.3
The vector space Rn
Definition 1 (Vectors in Rn ). A vector is an ordered list of n numbers xj , j = 1, 2, . . . , n, for some

positive integer n, which is called the dimension of the vector. The integers j = 1, 2, . . . , n that order
the list are called the indices, and the corresponding numbers xj are called the entries. That is, for
each j = 1, 2, . . . , n, xj is the jth entry on the list. The set of all n dimensional vectors is denoted
by Rn .
As for notation, we will generally use bold face to denote vectors: We write x Rn to say that
x is a vector in Rn . To write x out in terms of its entries, we often list the entries in a row, ordered
12
left to right so that the typical vector x R3 is

x = (x1 , x2 , x3 ) ,
where x1 , x2 and x3 are real numbers. When n is 2 or 3, it is often simpler to dispense with the
subscripts, and distinguish the entries by using different letters. In this way, one would write (x, y)
to denote a generic vector in R2 or (x, y, z) to denote a generic vector in R3 . We use 0 to denote the
vector in Rn with 0 in every entry.
Finally, we shall often consider sets of vectors {x1 , . . . , xm } in Rn where the different vectors are
distinguished by subscripts. A subscript on a boldface variable such as xj always indicates the jth
vector in a lsit of vectors and not the jth entry of a vector x. When we need to refer to the kth entry
of xj , we shall write (xj )k .
Definition 1 may not be what you expected. After all, we began this chapter discussing three
dimensional vectors that were defined as quantities with length and direction. When the term
vector was coined, people had in mind the description of the position and motion of points in three
dimensional physical space. For such vectors, the length and the direction have a clear geometric
meaning.
But what about vectors like (x, y, z, , , ), which belongs to R6 ? What would we mean by the
length of such a vector, and what would we mean by the angle between two such vectors?
Perhaps surprisingly, there is a useful notion of length and direction in any number of dimensions.
But until we define direction and magnitude, we cannot use these notions to define vectors themselves! Therefore, the starting point is the definition of vectors in Rn as ordered lists of n real
numbers.
Rn is more than just a set; it comes equipped with an algebraic structure that makes it what
is known as a vector space. The algebraic structure consists of two algebraic operations: scalar
multiplication and vector addition. As we have already stated, Descartes idea had such an enormous
impact because it brought together what had been two quite separate branches of mathematics
algebra and geometry. Our plan for the rest of this section is to develop the algebraic aspects of
Descartes idea, and then show how the algebra may be leveraged to apply our geometric intuition
about three dimensional vectors to vectors of any dimension.
Definition 2 (Scalar Multiplication). Given a number a R and a vector x = (x1 , x2 , . . . , xn ),
define the product of a and x, denoted ax, is defined by
ax = (ax1 , ax2 , . . . , axn ) .
For any vector x, x denotes the product of 1 and x.
Example 1 (Multiplying numbers and vectors). Here are several examples:
2( 1, 0, 1) = ( 2, 0, 2)
( 1/2, 1/2) = ( /2, /2) = (/2, /2)
By
useful, we mean useful for solving equations, among other things In other words, useful in a practical sense,
even in, say, eight dimensions.
13
0(a, b, c) = (0, 0, 0) = 0 .
Definition 3 (Vector Addition). Given two vectors x and y in Rn for some n, define their vector
sum, x + y, by summing the corresponding entries:
x + y = (x1 + y1 , x2 + y2 , . . . , xn + yn ) .
We define the vector difference of x and y, x y by x y = x + (y).
Note that vector addition does not mix up the entries of the vectors involved at all: For each j,
(x + y)j = xj + yj .
The third entry, say, of the sum depends only on the third entries of the summands.
For this reason, vector addition inherits the commutative and associative properties of addition in
the real numbers. It is just the addition of real numbers done in parallel.
That is: vector addition is commutative, meaning that x + y = y + x and associative, meaning
that (x + y) + z = x + (y + z). In the same way, one sees that scalar multiplication distributes over
vector addition:
a(x + y) = (ax) + (by) .
Example 2 (Vector addition).
( 3, 2, 5) + (1, 1, 1) = (2, 3, 6)
(8, 2, 4, 12) + (0, 0, 0, 0) = (8, 2, 4, 12)
(8, 2, 4, 12) + ( 8, 2, 4, 12) = (0, 0, 0, 0) = 0 .
There is a geometric way to think about vector addition in R2 . Identify the vector (x, y) R2
with the point the Euclidean plane having these Cartesian coordinates. We can then represent this
vector geometrically by drawing an arrow with its tail at the origin and its head at (x, y). The
following diagram shows three vectors represented this way: x = ( 1/2, 1/2), y = (3/2, 1) and their
sum, x + y = (1, 3/2).
14

The vectors x, y and x + y themselves are drawn in bold. There are also two arrows drawn more
lightly: one is a parallel copy of x transported so its tail is at the head of y. The other is a parallel
copy of y transported so its tail is at the head of x. These four arrows run along the sides of the
parallelogram whose vertices are the origin, and the points corresponding to x, y and x + y. As you
see, the arrow representing x + y is the diagonal of this parallelogram that has its tail end at the
origin.
A similar diagram could be drawn for any pair of vectors and their sum, and you see that we
can think of vector addition in the plane as corresponding to the following operation:
Represent the vectors by arrows as in the diagram. Transport one arrow without turning it that
is, in a parallel motion to bring its tail to the other arrows head. The head of the transported arrow
is now at the point corresponding to the sum of the vectors.
Example 3 (Subtraction of vectors). Let x and y be two vectors in the plane R2 , and let w = x y.
Then, using the associative and commutative properties of vector addition,
x = x + (y y) = (x y) + y = y + w .
Using the same diagram, with the arrow labeled a bit differently, we see that w = x y is the arrow
running from the head of y to the head of x, parallel transported so that its tail is at the origin.
Now let us begin to connect the algebra we have developed in this subsection with Descartes
ideas. The key is the introduction of the standard basis for Rn :
Definition 4 (Standard basis for Rn ). For j = 1, . . . , n, let ej denote the vector in Rn whose jth
entry is 1, and all of whose remaining entries are 0. The ordered set {e1 , . . . , en } is the standard
basis for Rn .
For example, if n = 3, we have
e1 = (1, 0, 0)
e2 = (0, 1, 0)
and
e3 = (0, 0, 1) .
In this three dimensional case, subscripts are often more of a hinderance than a help and a standard
notation is
i = e1
j = e2
and
k = e3 .
15
Definition 5 (Linear combination). Let {v1 , . . . , vm } be any set of m vectors in Rn . A linear

combination of these vectors is any expression of the form
m
X
aj xj .
j=1
Theorem 1 (Fundamental property of the standard basis). Every x = (x1 , . . . , xn ) in Rn can be

expressed as a linear combination of the standard basis vectors {e1 , . . . , en }, and the coefficients are
uniquely determined:
x=
n
X
xj e j .
(1.12)
j=1
Proof: By definition,
n
X
xj ej = x1 (1, 0, . . . , 0) + + xn (0, 0, . . . , 1) = (x1 , x2 , . . . , xn ).
j=1
Thus, any vector x = (x1 , x2 , . . . , xn ) can be written as a linear combination of the standard
n
X
yj ej = (y1 , y2 , . . . , yn ).Thus,
basis vectors: Next, by the computation we just made,
j=1
x=
n
X
yj ej yj = xj
for each j = 1, . . . , n ,
j=1
and hence the coordinates are uniquely determined.

The fact that every vector in Rn can be expressed as a unique linear combination of the standard
basis vectors is a special property of this set. For many other sets sets of vectors in Rn , there may
be vectors that cannot be expressed as a linear combination at all, while others can be expressed as
such in infinitely many ways.
Example 4. Let n = 2 and let v0 = (3, 4). v1 = (1, 1), v2 = (1, 1) and v3 = (0, 2). Then as you
can check,
3
3
7
1
v1 + v2 + 2v3 = 3v1 + v3 = 3v2 + v3 .
2
2
2
2
Thus, in this case, v0 can be expressed as a linear combination of {v1 , v2 , v3 }, and there are several
v0 =
ways to choose the coefficients {a1 , a2 , a3 } in fact, there are infinitely many.
One the other hand, if v0 = (1, 1, 1) and v1 = (1, 1, 0), v2 = (1, 0, 1) and v3 = (0, 1, 1),
then for any numbers a, b and c, we have
av1 + bv2 + cv3 = (a b, c a, b c) .
Adding up the entries on the right we get a b + c a + b c = 0. Adding up the entries in v0 , we
get 3. Hence there is no choice of a, b and c for which v0 = av1 + bv2 + cv3 . Thus, in this case, v0
cannot be expressed as a linear combination of {v1 , v2 , v3 }.
The standard basis vectors thus provide the analog of a Cartesian frame for Rn , in that one can
get to any vector in Rn by adding up a multiples of the standard basis vectors, just as one can get to
any point by moving along the directions of the vectors in the frame. However, frames were defined
in terms of orthogonality, and so far we have no notion of geometry in Rn , only algebra. In the next
subsection we bring in the geometry.
16
1.1.4
Geometry and the dot product
So far, we have considered Rn in purely algebraic terms. Indeed, the modern notion of an abstract
vector space is a purely algebraic construct generalizing the algebraic structure on Rn that has been
the subject of the last subsection.
Additional structure is required to make contact with the notions of length and direction that
have been traditionally associated to vectors. Let us begin with length, recalling that we have
identified the length of a vector with distance from the origin. We introduce the notion of a metric
which measures the distance between two points.
Definition 6. Let X be a set. A function % on the Cartesian product X X with values in [0, )
is a metric on X in case:
(1) %(x, y) = 0 if and only if x = y.
(2) For all x, y X, %(x, y) = %(y, x).
(3) For all x, y, z X,
%(x, z) %(x, y) + %(y, z) .
(1.13)
When % is a metric on X, the pair (X, %) is called a metric space.

As noted above, a metric measures the distance between two points in a metric space. At some
intuitive level, the distance corresponds to the length of the shortest path connecting x and y.
On an intuitive level, the phrase in quotes motivates the three items in the definition: Two points
are the same if and only if there is no distance between them. This leads to (1). The distance from
x to y is the same as the distance from y to x; just go back on the same path. This leads to (2).
Finally, if you insist on stopping by y on your way from x to z, the detour can only increase the total
distance traveled. This leads to (3).
You might be able to think of some more requirements you would like to impose on the concept
of distance. However, the mathematical value of a definition lies in the applicability of the theorems
one can prove using it. It turns out that the definition of metric that we have just given provides a
framework in which one can prove a great many very useful theorems. It is a very fruitful abstraction
of the notion of distance in physical space. We now prepare to introduce the Euclidean metric on
Rn .
Definition 7 (Euclidean distance). The Euclidean length of a vector x = (x1 , . . . , xn ) Rn is
denoted by kxk, and is defined by
1/2
n
X
kxk =
x2j
.
j=1
n
The distance between two vectors x, y R is defined by kx yk. A vector x Rn is called a unit
vector in case kvxk = 1; i.e., in case x has unit length.
We sometimes think of unit vectors as reprinting pure directions . Given any non-zero vector
1
x, we can write define the unit vector u =
x, which is called the normalization of x, and then
kxk

we have
17

x = kxk
1
x
kxk

= kxku .
This way of writing x expresses it as the product of its length and direction.
As you can check from the definition, for any x Rn and any t R,
ktxk = |t|kxk .
(1.14)
As we shall soon see, the function %E defined by

%E (x, y) := kx yk
(1.15)
is a metric on Rn , and is called the Euclidean metric. Indeed, with this definition of %E on R2 , the
p
distance between two vectors (x, y) and (u, v) is (x u)2 + (y v)2 . which is of course the usual
formula derived from the Pythagorean Theorem.
It is easy to see that the function %E defined in (1.15) satisfies requirements (1) and (2) in
the definition of a metric. The fact that it also satisfies (3) is less transparent, but fundamentally
important.
The first step towards this is to write kxk in terms of the dot product, which we now define:
Definition 8 (Dot product). The dot product of two vectors
a = (a1 , . . . , an ) , b = (b1 , . . . , bn )
in Rn is given by
a b = a1 b1 + a2 b2 + an bn .
Note that the dot product is commutative, meaning that a b = b a for all a, b Rn . This
follows directly from the definition and the commutativity of multiplication in R. In the same way
one sees that the dot product distributes in the sense that for all a, b, c Rn , and all s, t R,
(sa + tb) c = s(a c) + t(b c) .
However, except when n = 1, it does not make sense to talk about the associativity of the dot
product: For n > 1 and a, b, c Rn , a b
/ Rn , so (a b) c is not defined.
From the definitions, we have kxk2 = x x. Therefore, using the distributive and commutative
properties of the dot product,
kx yk2
(x y) (x y)
= x x + y y 2x y ,
= kxk2 + kyk2 2x y .
(1.16)
The formula (1.16) has an interpretation in terms of the lengths of the vectors x and y, and the angle
between these vectors. The key to this is the law of cosines. Recall that if the lengths of the three
sides of a triangle in a Euclidean plane are A, B and C, and the angle between the sides with lengths
A and B is , then C 2 = A2 + B 2 2AB cos .
18

Now let x and y be any vectors in the plane R2 . Consider the triangle whose vertices are 0, x
and y. Define the angle between x and y to be the angle between the two sides of the triangle issuing
from the vertex at 0. Since the length of the side of the triangle opposite this vertex is kx yk, by
the law of cosines,
kx yk2 = kxk2 + kyk2 2kxkkyk cos .
Comparing this with (1.16), we conclude that x y = kxkkyk cos . Therefore, in two dimensions, we
have proved that the angle between two non-zero vectors x and y, considered as sides of a triangle
in the plane R2 , is given by the formula

= arccos
xy
kxkkyk

,
(1.17)
where the arccosine function is defined on [1, 1] with values in [0, ].

The same sort of reasoning applies to vectors in R3 since any two non-colinear vectors x and y
in R3 lie in the plane determined by the three points 0, x and y, and then the law of cosines may be
applied in this plane. Thus, the formula (1.17) is valid in R3 as well.
Why not go on from here? It may seem intuitively clear that just as in R3 , again in Rn for any
n 3, any two non-colinear vectors x and y lie in a two dimensional plane in which we can apply
the law of cosines, just as we did in R2 and R3 . This suggests that we use use (1.17) to define the
angle between two vectors in Rn :
Definition 9. Let x and y be two non-zero vectors in Rn . Then the angle between x and y is
defined to be

= arccos
xy
kxkkyk

,
(1.18)
Two vectors x and y in Rn are orthogonal in case x y = 0.

There is an important matter to be checked before going forward: Does this definition make
sense? The issue is that the arccos function is defined on [1, 1], so it had better be the case that
1
xy
1
kxkkyk
for all nonzero vectors x and y in Rn . This certainly is the case for n = 2 and n = 3, where we have
proved the formula (1.17) is true with the classical definition of . but what about larger values of
n? The following theorem shows that there is no problem with using (1.18) to define no matter
what the dimension n is. (It has many other uses as well!)
Theorem 2 (CauchySchwartz inequality). For any two vectors a, b Rn ,
|a b| kakkbk .
(1.19)
There is equality in (1.19) if and only if kbka = kakb

Proof: Clearly (1.19) is true, with equality, in case either of the vectors is the zero vector, and also
in this case kbka = kakb = 0.
19
Hence we may assume that neither a nor b is the zero vector. Under this assumption, define
a
b
x=
and y =
. Now let us compute kx yk2 :
kak
kbk
kx yk2 = (x y) (x y) = x x + y y 2x y = 2(1 x y) ,
where we have used the fact that x and y are unit vectors. But since the left had side is certainly
non-negative, is must be the case that x y 1.
Likewise, computing kx + yk2 = 2(1 + x y), we see that x y 1. Thus,
1 x y 1 ,
which is equivalent to (1.19) by the definition of x and y.
As for the cases of equality, |x y| = 1 if and only if either kx yk = 0 or kx + yk = 0, which
means x = y. From the definitions of x and y, this is the same as kbka = kakb.
Theorem 3 (Triangle inequality). For any three vectors x, y, z Rn ,
kx zk kx yk + ky zk .
(1.20)
Proof: Let a = x y and b = z y so that x z = a b. Then by (1.16), and then the

Cauchy-Schwarz inequality,
kx zk2 = ka bk2
= kak2 + kbk2 2a b
kak2 + kbk2 + 2kakkbk
(kak + kbk)2 .
Taking square roots of both sides, and recalling the definitions of a and b, we obtain (1.20).
Theorem 4. The Euclidean distance function
p
%E (x, y) := (x y) (x y) = kx yk
is a metric on Rn .
Proof: The function %E (x, y) is non-negative and clearly satisfies conditions (1) and (2) in the
definition of a metric. By Theorem 3, it also satisfies (3).
1.1.5
Parallel and orthogonal components
Now that we have equipped Rn with geometric as well as algebraic structure, let us put this to work
for something that is simple but useful the decomposition of vectors into parallel and orthogonal
components:
Definition 10 (Parallel and orthogonal components). Given some non-zero vector a Rn , let
1
u :=
a, which is the unit vector in the direction of a. We can decompose any vector x Rn into
kak
two pieces, xk and x where
xk := (x u)u
and
x := x (x u)u .
(1.21)
20
These two vectors are called, respectively, the parallel and orthogonal components of x with respect
to a.
By the definition and simple computations,
x = xk + x
and
xk x = 0 .
(1.22)
As a consequence of (1.21),
kxk2 = (xk + x ) (xk + x ) = kxk k2 + kx k2 ,
which is a form of the Pythagorean Theorem.
Note that xk is a multiple of u, and hence of a, which is why we refer to xk as parallel to
a. (Actually, it could be a negative multiple of a, in which case xk and a would have the opposite
direction, but would at least lie on the same line through the origin.) Also x is orthogonal to xk ,
and hence to a, which is why we refer to x as orthogonal to a.
There is one final part of our terminology to be justified: the definite article the: Suppose
x = b + c where b is a multiple of a, and c is orthogonal to a. Then from this way of writing x as
well as (1.22) we have b + c = xk + x which can be rearranged into
b xk = x c .
Note that the vector on the left is a multiple of a, while the vector on the right is orthogonal to a.
Since they are both the same vector, they are orthogonal to themselves. But the only vector that
is orthogonal to itself is 0, and so b xk = 0 and x c = 0. That is, b = xk and c = x . In
summary, there is one and only one way to decompose a vector x as a sum of multiple of a and a
vector orthogonal to a, and this decomposition is given by (1.21)
The decomposition of vectors into parallel and orthogonal components is often useful. Here is a
first example of this.
Example 5. A 100 pound weight sits on an slick (frictionless) incline making a 60 degree angle with
the horizontal. It is held in place by a rope attached to the base of the weight and tied down at the
top of the ramp. The tensile strength of the rope is such that it is only guaranteed not to break for
tensions of no more than 80 pounds. Is this a dangerous situation?
21
To answer this we need to compute the tension in the rope. Let us use coordinates with the rope
lying in the x, y plane, and the y axis being vertical. The gravitational force vector, measured in
pounds, is
f = (0, 100) .
The unit vector pointing down the slope is
1
u = (cos(/3), sin(/3)) = (1, 3)
2
since 60 degrees is /3 radians. The tension in the rope must balance the component of the gravitation
force in the direction u; i.e., the direction of possible motion. That is, the magnitude of the tension
will be kfk k where fk is computed with respect to u. Doing the computation we find
fk = (f u)u = 25 3(1, 3) ,
and thus kfk k = 50 3 86.6. Look out below!

Here is another way to think about the computation in the previous example. The gravitational
force vector f has a simple expression in standard x, y coordinates, but these coordinates are no well
adapted to the problem at hand since neither coordinate axis corresponds to a possible direction of
possible motion. The direction of possible motion is given by u.
Let us consider a coordinate system built around the direction of u. We then take v to be on of
the two unit vectors in R2 that is orthogonal to u. Since f points downward, we take the one that
1
points downward. That is bv = ( 3, 1). As you can easily check
2
uu=vv =1
and
uv =0 ,
(1.23)
so that {u, v} is an orthogonal pair of unit vectors.

Let us write the force vector f in coordinates based on the {u, v} frame of reference. That is,
f = uu + vv
(1.24)
for some numbers u and v, which are the coordinates of f with respect to this frame of reference.
The u, v coordinates of f are directly relevant to our problem. In particular u is the magnitude of the
force in the direction u, and is what the tension in the rope must balance. Hence in these coordinates,
our question becomes: Is u > 80?.
To answer this question we need to know how to compute u and v in terms of the x, y coordinates
of f , which is what we are given. Here is how: Take the dot products of both sides of (1.24) with u
and v:
f u = (uu + vv) u = uu u + vv v = u
where we have used the distributive property of the dot product and (1.23). In the same way, we
find v = f v. Thus, we can re-write (1.24) as
f = (f u)u + (f v)v ,
or in other words, u = f u and v = f v.
(1.25)
22

As we shall see:
The first step in solving many problems is to introduce a system of coordinates that is adapted to
the problem, and in particular is built out of directions given in the problem.
The most broadly useful and convenient coordinate systems in Rn are those constructed using a
set of n mutually orthogonal unit vectors, such as the set {u, v} of orthogonal unit vectors in R2 that
we have just used to build coordinates for our inclined plane problem. The next subsection develops
this idea in general.
1.1.6
Orthonormal subsets of Rn
Definition 11 (Orthonormal vectors in Rn ). A set {u1 , . . . , um } of m vectors in Rn is orthonormal

in case for all 1 i, j m
ui uj =
i=j
i 6= j
(1.26)
Example 6. The set of standard basis vectors {e1 , . . . , en } is one very simple example of an orthonormal set. Also, any non-empty subset of an orthonormal set is easily seen to be orthonrmal,
so we can get other examples by taking non-empty subsets of {e1 , . . . , en }. These are important
examples, but not the only ones. Here is a more invteresting example: Let
u1 =
1
(1, 2, 2)
3
u2 =
1
(2, 1, 2)
3
and
u3 =
1
(2, 2, 1) .
3
(1.27)
Then you can easily check that (1.26) is satisfied, so that {u1 , u2 , u3 } is an orthonormal set in R3 .
In fact, it is a rather special kind of orthonormal set: It is an orthonormal basis, as defined next.
The main theorem concerning orthonormal sets in Rn is the following.
Theorem 5 (Fundamental Theorem on Orthonormal Sets in Rn ). Let {u1 , . . . , un } be any orthonormal set in Rn consisting of exactly n vectors. Then every vector x Rn can be written as a linear
combination of the the vectors {u1 , . . . , un } in exactly one way, namely
x=
n
X
(x uj )uj .
(1.28)
j=1
Moreover, the squared length of x is the sum of the squares of the coefficients in this expansion:
kxk2 =
n
X
(x uj )2 .
j=1
The standard basis of Rn , {e1 , . . . , en } is a set of n orthonormal vectors in Rn , and so the

theorem says that
x=
n
X
j=1
xj ej =
n
X
(x ej )ej
j=1
is the unique way to express any x in R as a linear combination of the standard basis vectors. This
is the content of Theorem 1. Theorem 5 generalizes this to arbitrary sets of n orthonormal vectors in
23
Rn . It allows us to take any set of n orthonormal vectors in Rn as the basis of a coordinate system
in Rn . This will prove to be very useful in practice. It will allow us to use coordinates that are
especially adapted to whatever computation we are trying to make, which will often be much simpler
than a direct computation using coordinates based on the stanard basis. The next definitions pave
the way for this.
Definition 12 (Orthonormal basis). An orthonormal basis in Rn is any set of n orthonormal vectors
in Rn .
As we have noted above, the standard basis is one example, but there are many others. We have
seen in Example 6 that {u1 , u2 , u3 } with the vectors specified by (1.27) is an orthonormal set of three
vectors in R3 . Hence it is an orthonormal basis for R3 .
Definition 13 (Coordinates with respect to an orthonormal basis). Consider an orthonormal basis
{u1 , . . . , un } of Rn , and a vector x in Rn . Then the numbers x uj , 1 j n, are called the
coordinates of x with respect to {u1 , . . . , un }.
The coordinates of a vector x Rn with respect to an orthonormal basis behave just like
Cartesian coordinates in that they tell you how to get from 0 to x: If the coordinates are y1 , . . . , yn ,
start at 0, move y1 units in the u1 direction, then move y2 units in the u2 direction and so forth.
The hard part of the proof of Theorem 5 is contained in the following lemma:
Lemma 1 (No n + 1 orthonormal vectors in Rn ). There does not exist any set of n + 1 orthonormal
vectors in Rn .
The proof of this simple statement is very instructive, and very important, but somewhat involved. We give it in full in the next subsection. For now, let us take it on faith, and see how we
may use it to prove Theorem 5. Most of the applications we make in the next section will be in R2
and R3 , and you will probably agree that the lemma is geometrically obvious in these cases where
you can easily visualize things.
Proof of Theorem 5: Let {u1 , . . . , un } be any set of n orthonormal vectors in Rn , and let x be
ano non-zero vector in Rn . Define the vector z by
z := x
n
X
(x uj )uj .
j=1
for each i = 1, . . . , n, we have
n
n
X
X
z ui = x
(x uj )uj ui = x ui
(x uj )uj ui = x ui x ui = 0 .
j=1
j=1
Thus, z is orthogonal to each ui .
1
z. Since this
kzk
unit vector is orthogonal to each unit vector in the the orthonormal set {u1 , . . . , un }, the augmented
Now suppose that z 6= 0, Then we may define a unit vector un+1 by un+1 =
set {u1 , . . . , un , un+1 } would be a set of n + 1 orthonormal vectors in Rn . By Lemma 1, this is
24
impossible. Therefore, z = 0. By the definition of z, this means that x =
n
X
(x uj )uj , which is
j=1
(1.28). Thus, every vector x Rn can be written as a linear combination of the vectors {u1 , . . . , un }.
n
X
Moreover, there is only one way to do this, since if x =
yj uj , taking the dot product of both sides
j=1
with ui yields
x ui =
n
X
yj uj ui =
j=1
n
X
yj (uj ui ) = yi .
j=1
That is, each yi must equal x ui .

Finally, going back to (1.28), we compute
n
X
kxk2 = x x = (x uj )uj
j=1
n
X
!
(x uk )uk
k=1
(x uj )(x uk )uj uk =
j,k=1
1.1.7
n
X
n
X
(x uj )2 .
j=1
Householder reflections and orthonormal bases
In this subsection we shall prove Lemma 1. The proof is very interesting, and introduces many
techniques and ideas that will be important later on.
We begin by introducing an extremely useful class of functions f from Rn to Rn : the Householder
reflections.
First, for n = 2, fix a unit vector u R2 and consider the line ù through the origin that is
orthogonal to u. Then, for any x R2 , define hu (x) to be the mirror image of x across the line ù .
That it, hu (x) is the reflection of x across the line ù . Here is a picture illustrating the transformation
form x to hu (x):
The transformation from x to hu (x) is geometrically well defined, and you could easily plot the
output point hu (x) for any given input point x. But to do computations, we need a formula. Let us
derive a formula.
The key thing to realize, which you can see in the picture, is that both x and hu (x) have the
same component orthogonal to u (that is, along the line ù ) and have opposite components parallel
25
to u. In formulas, with respect to the direction u,

(hu (x)) = x
and
(hu (x))k = xk .
Therefore, since hu (x) = (hu (x)) + (hu (x))k , we have the formula
hu (x) = x xk .
(1.29)
Then since x = x (x u)u and xk = (x u)u, we deduce the more explicit formula
hu (x) = x 2(x u)u .
(1.30)
We have derived the formula (1.30) for n = 2. However, the formula does not explicitly involve
the dimension and makes sense in any dimension. Now, given any unit vector u Rn , for any positive
integer n, we use this formula to define the transformation hu from Rn to Rn that we shall call the
Householder reflection on Rn in the direction u:
Definition 14 (Householder reflection in the direction u). For any unit vector u Rn , the function
hu from Rn to Rn is defined by (1.30).
1
Example 7. Let u = ( 1, 1, 1). As you can check, this is a unit vector. Let x = (x, y, z)
3
denote the generic vector in R3 . Let us compute hu (x). First, we find
xu=
yxz
and hence
(x u)u =
yxz
( 1, 1, 1) .
3
Notice that the square roots have (conveniently) gone away! Now, from the definition of hu ,
hu (x, y, z) = (x, y, x)
2y 2x 2z
1
( 1, 1, 1) = (x + 2y 2z, 2x + y + 2z, 2x + 2y + z) .
3
3
This is an example of a function, or, what is the same thing, transformation from R3 to R3 . If the
input vector is (x, y, z), the output vector is

x + 2y 2z 2x + y + 2z 2x + 2y + z
hu (x, y, z) =
,
,
.
3
3
3
(1.31)
To conclude the example, let us evaluate the transformation at particular vector. We choose,
more or less at random,
x = (1, 2, 3) .
Plugging this into our formula (1.31) we find
hu (1, 2, 3) =
1
( 1, 10, 5) .
3
Let us compute the length of hu (x):

khu (x)k =
Notice that kxk =
1+4+9=
1
1
k(1, 10, 5)k =
1 + 100 + 25 = 14 .
3
3
14, so khu (x)k = kxk. This will always be the case, as we explain
next and as should be the case: reflection preserves the lengths of vectors.
26

Householder reflections have a number of special properties that will prove to be very useful to
us. Here is the first of these:

Lemma 2 (Householder reflections preserve dot products). For any two vectors x and y in Rn , and
any unit vector u in Rn ,
(hu (x)) (hu (y)) = x y .
(1.32)
khu (x)k = kxk .
(1.33)
In particular,
Proof: We use (1.29) and the fact that x is orthogonal to yk and that xk is orthogonal to y to
compute:
(hu (x)) (hu (y))

x xk y yk
x y + xk yk x yk xk y
x y + xk yk ,
and
xy

x +xk y + yk
= x y + xk yk + x yk + xk y
= x y + xk yk ,
Comparing the two computations proves (1.32). Then (1.33) follows by considering the special
case y = x.
In fact, since angles between vectors as well as the lengths of vectors are defined in terms of the
dot product, Householder reflections preserve angles between vectors as well as the lengths of vectors,
as you would expect from the diagram. In particular, Householder reflections preserve orthogonality.
Householder reflections are invertible transformations. In fact, they are their own inverses: For
all x Rn , hu (hu (x)) = x, That is, hu hu is the identity function.
To see this from the defining formula, we compute
hu (hu (x)) = hu (x xk ) = x (xk ) = x + xk = x .
That is, reflecting a vector twice (about the same direction) leaves you with the vector you started
with.
Since reflection does not alter the length of a vector, if we are given vectors x and y with
kxk =
6 kyk, then we cannot possibly find a unit vector u such that hu (x) 6= y. However, if kxk = kyk,
but x 6= y, then there is always a standard Householder reflection hu such that hu (x) = y:
Lemma 3. Let x, y Rn , n 2, satisfy kxk = kyk, but x 6= y. Then there is a unit vector u Rn
so that for the corresponding Householder reflection hu ,
hu (x) = y .
27
In particular, one may always choose

u=
1
(x y) .
kx yk
(1.34)
Moreover, with this choice of u, for any z Rn that is orthogonal to both x and y
hu (z) = z .
(1.35)
Proof: Define u by (1.34), and compute

2(x u)u =
2x (x y)
(x y) .
kx yk2
(1.36)
Since kxk = kyk,

kx yk2 = kx|2 + kyk2 2x y = 2(kxk2 x y) = 2x (x y) .
Therefore, from (1.36) we have 2(x u)u = x y, and so hu (x) = x (x y) = y.
The final part is simple: If z is orthogonal to both x and y, then it is orthogonal to u, and then
(1.35) follows from the definition of hu .
1
(1, 2, 2) and y = e1 = (1, 0, 0). These are both unit vectors, and hence
3
kxk = kyk, so there is a unit vector u such that hu (x) = y, and u is given by (1.34). We compute
Example 8. Let x =
u:
xy =
1
1
2
(1, 2, 2) (1, 0, 0) = [(1, 2, 2) (3, 0, 0)] = (1, 1, 1) .
3
3
3
Normalizing, we find
1
u = (1, 1, 1) .
3
Now simple computations verify that, as claimed, hu (x) = y.
We are now ready to prove Lemma 1, which says that there does not exist any orthonormal set
of n + 1 vectors in Rn .
Proof of Lemma 1. We first observe that for n = 1, there are exactly two unit vectors, namely (1)
and ( 1). Since these vectors are not orthogonal, there are exactly two orthonormal sets in R1 ,
namely {(1)} and {(1)}, and each consists of exactly one vector. This proves the Lemma for n = 1.
We now proceed by induction. For any n 2 we suppose it is proved that there does not exist
any orthonormal set of n vectors in Rn1 . We shall show that it follows from this that there does
not exist any orthonormal set of n + 1 vectors in Rn .
Suppose on the contrary that {u1 , . . . , un+1 } is an orthonormal set of vectors in Rn . Then there
exists an an orthonormal set {v1 , . . . , vn+1 } of vectors in Rn such that vn+1 = en . To see this, note
that if un+1 = en , we already have the desired orthonormal set. Otherwise, by Lemma 3 there exists
a unit vector in Rn such that
hu (un+1 ) = en .
28
Then, since householder reflections preserve lengths and orthogonality, if we define vj = hu (uj ),
j = 1, . . . , n + 1 {v1 , . . . , vn+1 } is also an orthonormal set in Rn , and by construction, vn+1 = en .
Therefore, for each j = 1, . . . , n,
0 = vj vn+1 = vj en .
Since vj en is simply the final entry of vj , this means that for each j = 1, . . . , n, vj has the form
vj = (wj , 0)
where wj is a unit vector in Rn1 .
Since {v1 , . . . , vn } is orthonormal in Rn , {w1 , . . . , wn } is orthonormal in Rn1 , since the final
zero coordinate simply goes along for the ride. However, this is impossible, since we know that
there does not exist any orthonormal set of n vectors in Rn1 . We arrived at this contradiction by
assuming that there existed and orthonormal set of n + 1 vectors in Rn . Hence this must be false.
1.2
Lines and planes in R3
In this section we shall study the geometry of lines and planes in R3 . We shall see that if we
use coordinates based on a well-chosen chosen orthonormal basis, it is very easy to compute many
geometric qunatities such as, for example, the distance between two lines in R3 . Of course, to do this,
we need a systematic method for constructing orthonormal bases. In R3 , the cross product provides
such a method, and has many other uses as well. In the next subsection, we introduce the cross
product, starting from a question about area that the cross product is designed to answer.
1.2.1
The cross product in R3
Let a and b be two vectors in R3 , neither a multiple of the other, and consider the triangle with
vertices at 0, a, b, which naturally lies in the plane through these three points. The cross product
gives the answer to the following question, and a number of other geometric questions as well:
How can we express the area of this triangle in terms of the Cartesian coordinates of a and b?
The classical formula for the area of a triangle in a plane is that it is one half the length of the
base times the height. Let us take the side running from 0 to a as the base, so that the length of the
base is kak. Then, using to denote the angle between a and b, the height is kbk sin . Thus, the
area A of the triangle is
A :=
1
kakkbk sin .
2
(Note that since, by definition, [0, ] sin 0.)

Using the identity sin2 + cos2 = 1, we can write this as
4A2 = kak2 kbk2 sin2 = kak2 kbk2 (1 cos2 ) = kak2 kbk2 (a b)2 .
Now calculate the right hand side, taking a = (a1 , a2 , a3 ) and b = (b1 , b2 , b3 ).
LINES AND PLANES IN R3
1.2.
29
We find, after a bit of algebra,

kak2 kbk2 (a b)2 = (a2 b3 a3 b2 )2 + (a3 b1 a1 b3 )2 + (a1 b2 a2 b1 )2 .
The square root of the right hand side is the twice area in question. Notice that the right hand side
is also square of the length of a vector in R3 , namely, the vector a b, defined as follows:
Definition 15 (Cross product in R3 ). Let a and b be two vectors in R3 . Then the cross product of
a and b is the vector a b where
a b = (a2 b3 a3 b2 )e1 + (a3 b1 a1 b3 )e2 + (a1 b2 a2 b1 )e3 .
(1.37)
Example 9. Computing from the definition, we find

e1 e2 = e3
e2 e3 = e1
and
e3 e1 = e2 .
(1.38)
By the computations that led to the definition, we have that

ka bk = kakkbk sin .
(1.39)
This tells us the magnitude of a b. What is its direction? Before dealing with this geometric
question, it will help to first establish a few algebraic properties of the cross product.
Notice from the defining formula (1.37) that
a b = b a .
Thus the cross product is not commutative; instead, it is anticommutative. In particular, for any
a R3 ,
a a = a a = 0 .
(1.40)
Also, introducing a third vector c = (c1 , c2 , c3 ), we have from the definition that
a (b + c)
=
[a2 (b3 + c3 ) a3 (b2 + c2 ))]e1 + [a3 (b1 + c1 ) a1 (b3 + c3 )]e2 + [a1 (b2 + c2 ) a2 )b1 + c1 )]e3
(a2 b3 a3 b2 )e1 + (a3 b1 a1 b3 )e2 + (a1 b2 a2 b1 )e3
(a2 c3 a3 c2 )e1 + (a3 c1 a1 c3 )e2 + (a1 c2 a2 c1 )e3
= ab+ac .
Thus, a (b + c) = a b + a c, which means that the cross product distributes over vector
addition. From this identity and the anticommutivity, we see that (b + c) a = b a + c a; i.e.,
the distributivitiy holds on both sides of the product.
Finally, a similar but simpler proof shows that for any number t, (ta) b = t(a b) = a (tb).
We summarize our conclusions in a theorem:
Theorem 6 (Algebraic properties of the cross product). Let a, b and c be any three vectors in R3 ,
and let t be any number. Then
30
a b = b a .
a (b + c) = a b + a c .
(ta) b = t(a b) = a (tb) .

We now return to the geometric information contained in the cross product. The following result,
which relates the cross product and the dot product, is useful for this.
Theorem 7. Let a, b, c R3 . Then
(a b) c = a (b c) .
Proof: We compute:
(a b) c
(a2 b3 a3 b2 )c1 + (a3 b1 a1 b3 )c2 + (a1 b2 a2 b1 )c3
(b2 c2 b3 c2 )a1 + (b3 c1 b1 c3 )a2 + (b1 c2 b2 c1 )a3
(b c) a
= a (b c) .
Therefore, by (1.40) and Theorem 7, for any a, b R3 ,

(a b) b = a (b b) = 0 .
Likewise, using also the anticommutivity of the cross product,
(a b) a = (b a) a = b (a a) = 0 .
We have proved:
Theorem 8. Let a, b R3 . Then a b is orthogonal to both a and b.
Let v1 and v2 be two vectors such that neither is a multiple of the other. Then by Theorem 8,
v1 v2 is a non-zero vector orthogonal to every vector of the form
sv1 + tv2
s, t R ,
which is to say that a := v1 v2 is orthogonal to every vector in the plane in R3 determined by

(passing through) the 3 points 0, v1 and v2 . In other words:
The cross product v1 v2 gives the direction of the normal line to the plane through 0, v1 and v2 ,
provided these are non-colinear, so that they do determine a plane.
Theroem 8 says when a b 6= 0, the unit vector
1
ab
ka bk
is one of the two unit vectors orthogonal to the plane through 0, a and b. Which one is it?
1.2.
31
Definition 16. An orthonormal basis {u1 , u2 , u3 } of R3 is a right-handed orthonormal basis in

case u1 u2 = u3 and is a left-handed orthonormal basis in case
u1 u2 = u3 .
Note that every orthonormal basis {u1 , u2 , u3 } of R3 is either left-handed or right-handed, since
u1 u2 must be a unit vector orthogonal to both u1 and u2 , so that u3 are the only possibilities.
Also note that the standard basis of R3 is right-handed by (1.38).
Theorem 9. Let {u1 , u2 , u3 } be any orthonormal basis of R3 . Then
u1 u2 = u3
u2 u3 = u1
u3 u1 = u2 .
(1.41)
In particular, {u1 , u2 , u3 } is right handed if and only if any one of the identities in (1.41) is valid,
and in that case, all of them are valid.
Proof: Suppose that u1 u2 = u3 . Then by Theorem 5,
u2 u3 = (u2 u3 u1 )u1 + (u2 u3 u2 )u2 + (u2 u3 u3 )u3 .
Since u2 u3 orthogonal to u2 and u3 , this reduces to
u2 u3 = (u2 u3 u1 )u1 ,
(1.42)
and we only need to compute u2 u3 u1 = u1 u2 u3 . By Theorem 7 and the hypothesis that

u1 u2 = u3 ,
u1 u2 u3 = u1 u2 u3 = u3 u3 = 1 .
Summarizing, under the assumption that u1 u2 = u3 , we have u2 u3 u1 = 1, and hence from
(1.42), we have
u1 u2 = u3 u2 u3 = u1 .
The same sort of computation also shows that
u1 u2 = u3 u3 u1 = u2 .
(1.43)
Indeed, by Theorem 5
u3 u1
(u3 u1 u1 )u1 + (u3 u1 u2 )u2 + (u3 u1 u3 )u3
(u3 u1 u2 )u2
We then compute
u3 u1 u2 = u3 u1 u2 = u3 u3 = 1 ,
from which we conclude (1.43). Thus, the first of the identities in (1.41) implies the other two. The
same sort of computations, which are left to the reader, show each of them implies the other two, so
that they are all equivalent.
Why is the distinction between right and left handed orthonormal bases useful? One consequence
of Theorem 9 is that one can use a formula just like (1.37) to compute the Cartesian components of
32
a b in terms of the Cartesian components of a and b for any coordinate system based on a any
right handed orthonormal basis, {u1 , u2 , u3 }, and not only the standard basis {e1 , e2 , e3 }.
Indeed, if we write a = a1 u1 + a2 u2 + a3 u3 , so that a1 , a2 and a3 are the Cartesian coordinates
of a with respect to the orthonormal basis {u1 , u2 , u3 }, and similarly for b, we find using Theorem 6
and Theorem 9 that
a b = (a2 b3 a3 b2 )u1 + (a3 b1 a1 b3 )u2 + (a1 b2 a2 b1 )u3 .
The next identity, for the cross product of three vectors, has many uses. For example, we shall
use it later on to deduce Keplers Laws from Newtons Universal Theory of Gravitation. It was for
exactly this purpose that Lagrange proved the identity, though he stated it in a different form.
Theorem 10 (Lagranges Identity). Let a, b, c R3 . Then
a (b c) = (a c)b (a b)c .
(1.44)
One way to prove Theorem 10 is to write a = (a1 , a2 , a3 ), b = (b1 , b2 , b3 ) and c = (c1 , c2 , c3 ), and
then to compute both sides of (1.44), and check that they are equal. It is much more enlightening,
and much less work, to do the computation using coorindates based on an orthonormal basis built
out of the given vectors a, b and c. This will be our first example of the strategy of using a well
adapted coordinate system. In the next subsection, we provide many more examples.
Proof of Theorem 10: Note first of all that if b is a multiple of c, then a (b c) = 0. So let
us assume for now that this is not the case. Let bk and b be the components of b that are parallel
and orthogonal to c respectively. Our assumption is that b 6= 0, and we can therefore build an
orthonormal basis {u1 , u2 , u3 } of R3 as follows: Define
u1 =
1
b ,
kb k
u2 =
1
c
kck
and
u3 = u1 u2 .
By the properties of the cross product, {u1 , u2 , u3 } is a right handed orthonormal basis for R3 . We
now compute
b c = b c = (kb ku1 ) (kcku2 ) = kb kkcku1 u2 = kb kkcku3 ,
where we have used Theorem 9 in the final step.
Now using Theorem 5, we write
a = (a u1 )u1 + (a u2 )u2 + (a u3 )u3 ,
and compute a (b c). By our computation of b c above, this gives
a (b c) = kb kkck[(a u1 )u1 u3 + (a u2 )u2 u3 + (a u3 )u3 u3 ],
and by Theorem 9 once more, this gives
a (b c)
= kb kkck[(a u1 )u2 + (a u2 )u1

=
(a c)b (a b )c .
(1.45)
1.2.
33
This is almost the formula we want. The final step is to observe that since bk is some multiple
of c, say, bk = tc, then
(a c)bk (a bk )c = t[(a c)c (a c)c] = 0 ,
(1.46)
and therefore the parallel component of b drops out so that ,

(a c)b (a b)c = (a c)b (a b )c .
(1.47)
Comparing (1.45) and (1.47) we see that (1.44) is true whenever b 6= 0. On the other hand, when
b = 0, (1.46) and the fact that in this case b c = b c = 0 show that (1.44) is true also when
b = 0.
The identity we get from Theorem 10 in the special case a = b is often useful:
Corollary 1. Let u be any unit vector in R3 . Then for all c R3
c = (u c) u
(1.48)
where c is the component of c orthogonal to b.

Note the resemblance of (1.48) to the formula ck = (u c)u.
Proof: Applying (1.44) with a = b = u, we obtain
u (u c) = (c u)u (u u)c = (c (c u)u) = c .
Now using the anticommutivity of the cross product, we obtain (1.48).
Thus, one can readily compute orthogonal components by computing cross products. The magnitude of c is even simpler to compute: Since b c = b c , and since kb c k = kbkkc k,
kc k =
kb ck
.
kbk
The cross product is not only useful for checking whether a given orthonormal basis {u1 , u2 , u3 }
is right handed or not; it is useful for constructing such bases. The next example concerns a problem
that often arises when working with lines and planes in R3 .
Example 10 (Constructing a right-handed orthonormal basis containing a given direction). Given
a nonzero vector v R3 , how can we find an orthonormal basis {u1 , u2 , u3 } in which
u3 =
1
v?
kvk
Here is one way to do it using the cross product. First, u3 is given. Let us next choose u1 . This
has to be some unit vector that is orthogonal to v.
If v = (v1 , v2 , v3 ) and vj = 0 for any j, then v ej = vj = 0, and we may take u1 = ej for this
j. On the other hand, if each vj is non-zero, define
w := e3 v = (v2 , v1 , 0) .
34
This is orthogonal to v by Theorem 8, and kwk =

6 0. Now define
u1 =
1
w.
kwk
Finally, define u2 = u3 u1 , so that u2 is orthogonal to both u3 and u1 , and is a unit vector. Thus,
{u1 , u2 , u3 } is an orthonormal basis, and since u3 u1 = u2 , Theorem 9 says that this basis is right
handed.
Now let us do this for specific vectors. Let v = (2, 1, 2). We will now find an orthonormal basis
{u1 , u2 , u3 } in which u3 points in the same direction as v.
1
1
v = (2, 1, 2). Next, since none of the entries of v is zero, we
For this, we must have u3 =
kvk
3
define
w := e3 v = ( 1, 2, 0),
and then
u1 :=
Finally we define
1
1
w = ( 1, 2, 0) ,
kwk
5
1
u2 := u3 u1 = ( 4, 2, 5) .
3 5
The method in the last example can be used to find a nice system of equation for the line through
two distinct points in R3 .
1.2.2
Equations for planes in R3
A plane in three dimensional space may be specified in several ways: One is by specifying a point x0
in the plane, and a direction vector a. The corresponding plane consists of all points x such that the
vector x x0 (which carries x0 to x) is orthogonal to a.
In this description, we refer to the vector x0 as the base point and the vector a as the normal
vector. Here is a diagram showing a plane with a given base point and normal vector, and also two
vectors x and y in the plane:
We can translate this verbal and pictorial description into an equation specifying the plane by
taking advantage of the fact that orthogonality can be expressed in terms of the dot product: The
vector x belongs to the plane if and only if
a (x x0 ) = 0 .
1.2.
35
Example 11 (The equation of a plane specified in terms of a base point and normal direction).
Consider the plane passing through the point x0 = (3, 2, 1) that is orthogonal to the vector a = (1, 2, 3).
Then x = (x, y, z) belongs to this plane if and only if (x x0 ) a = 0. Doing the computations,
((x, y, z) (3, 2, 1)) (1, 2, 3) = 0
x + 2y + 3z = 10 .
The solution set of the equation x + 2y + 3z = 10 consists of the Cartesian coordinates (with respect to
the standard basis) of the points in this plane. We say that x + 2y + 3z = 10 is an equation specifying
this plane.
As we have just seen in the last example, an equation specifying a plane can also be written in
the form a x = d. Indeed, defining d := a x0 ,
(x x0 ) a = 0
ax=d .
Example 12 (Parameterizing a plane specified by an equation). Consider the equation a x = d

where a = (1, 2, 1), and d = 10. Writing x = (x, y, z), the equation becomes x + 2y + z = 10.
Parameterizing solution sets means solving euations. Solving equations means eliminating variables. All three variables are present in this equation, so we can choose to eliminate any of them. Let
us eliminate z:
z = 10 2y x .
Thus, x = (x, y, z) belong to the plane if and only if
x = (x, y, 10 2y x) = (0, 0, 10) + x(1, 0, 1) + y(0, 1, 2) .
We have expanded the left hand side, and collected terms into a constant vector, a second constant
vector times x and a third constant vector times y. The point of this is that defining
x0 := (0, 0, 10) ,
v1 := (1, 0, 1)
and
v2 := (0, 1, 2) ,
we conclude that a vector x belongs to the plane if and only if it can be written in the form x0 + xv1 +
yv2 for some numbers x and y. Moreover, whenever x can be written in the form x0 + xv1 + yv2 ,
the numbers x and y are uniquely determined, since direct computation yields
x0 + sv1 + tv2 = (s, t, 10 2s t) .
If this is to equal (x, y, z), we must have s = x and y = t.
Thus there is a one-to-one correspondence between points x in the plane and vectors (s, t) R2
given by
x(s, t) = sv1 + tv2 .
As (s, t) varies over R2 , x(s, t) varies over the plane in question in a one-to one way. Thus, we have
parameterized the plane.
36
Later in this course, we shall be studying more general parametrized surfaces in R3 . Parameterized planes are our first example, and also one of the most important: One thing that is often useful
in solving problems involving more general parameterized surfaces is to compute parameterizations
of their tangent planes. Hence, what we are doing now is fundamental for what follows.
1.2.3
Equations for lines in R3
Given two distinct points x0 and x1 in R3 , define a = x1 x0 , and then for all t R, define
x(t) := x0 + ta :
As illustrated below, as t varies, x(t) varies over the set of points that one can reach starting
from x0 , and moving only in the direction of a (or its opposite).
We have arrived at a parameterization of the line without first finding an equation for it. We
now ask: What equation has this line as its solution set? One way to get an equation for the line is
to take the cross product of x(t) = x0 + ta with a: Since x(t) x0 = ta, and a a = 0,
a (x(t) x0 ) = 0 .
That is, every point on the line parameterized by x(t) = x0 + ta solves the equation
a (x x0 ) = 0 .
(1.49)
We now show that: A vector x R3 satisfies this equation if and only if it is on the line
parameterized by x(t) = x0 + ta.
1.2.
37
Having done this we shall have shown that the solution set of the equation (1.49) is exactly the
line parameterized by x(t) = x0 + ta. Since x0 can be any vector in R3 , and a can be any non-zero
vector in R3 , this shows that every line in R3 is given by an equation of the form (1.49) just as every
plane in R3 is given by an equation of the form a (x x0 ) = 0.
To see that the solution set of (1.49) is exactly the line parameterized by x(t) = x0 + ta, choose
a right handed orthonormal basis {u1 , u2 , u3 } in which u3 is a multiple of a. In Example 10, we
have shown how to construct such an orthonormal basis, but for our present purpose, we only need
to know that {u1 , u2 , u3 } exists.
We then expand x x0 with respect to this basis using Theorem 5:
x x0 = (u1 (x x0 ))u1 + (u2 (x x0 ))u2 + (u3 (x x0 ))u3 .
(1.50)
Taking the cross product with a = kaku3 , and using Theorem 9 we obtain:
a (x x0 )
= kak[(u1 (x x0 ))u3 u1 + (u2 (x x0 ))u3 u2 ]

= kak[(u1 (x x0 ))u2 (u2 (x x0 ))u1 ] .
By Theorem 5 once more,

ka (x x0 )k2 = kak2 [(u1 (x x0 ))2 + (u2 (x x0 ))2 ] .
Therefore, a (x x0 ) = 0 if and only if u1 (x x0 ) = 0 and u2 (x x0 ) = 0. But in this case,
(1.50) reduces to
x x0 = (u3 (x x0 ))u3 = (u3 (x x0 ))
1
a.
kak
Thus, a (x x0 ) = 0 if and only if x = x0 + ta where t = kak1 (u3 (x x0 )). In particular, as

claimed above, the solution set of (1.49) is exactly the line parameterized by x(t) = x0 + ta.
Defining d := a x0 , we can re-write (1.49) as
ax=d .
(1.51)
However, given two generic non-zero vectors a and d in R3 , it is not always the case that the
solution set of the equation a x = d is a line: It can also be the empty set. Indeed, for all x, a x
is orthogonal to both x and a. Hence, if d is not orthogonal to a, there is no vector x satisfying
a x = d. On the other hand, if d is orthogonal to a, then by (1.48) of the corollary to Theorem 10,
a (a d) = kak2 d
and hence
x0 :=
1
ad
kak2
is a solution of a x = d, and then evidently (1.51) is equivalent to (1.49) with this choice of x0 .
Given two non-zero vectors a and d in R3 , the equation a x = d has no solution if a is not
orthogonal to to a, and otherwise, its solution set is the line parameterized by
1
a d + ta .
kak2
38
Example 13 (Solving a x = d). Let a = (1, 2, 1) and d = ( 1/2, 1, 5/2). We check that
a d = 0, and hence the solutions set of a x = d is a line. We then compute kak2 = 6 and
a d = ( 6, 3, 0) .
Hence
1
1
a d) = ( 6, 3, 0) = (1, 1/2, 0) .
2
kak
6
Thus, the solution set is the line parameterized by
x0 :=
x(t) = x0 + ta = ( 1, 1/2, 0) + t(1, 2, 1) = (1 + t, 1/2 2t, t) .

Example 14 (The equation a x = d as a system of equations). Let a = (1, 2, 1) and b =
( 1/2, 1, 5/2) as in the previous example. Computing a x for x = (x, y, z), we find
(1, 2, 1) (x, y, z) = ( y 2z, x z, 2x + y) .
Hence, a x = d is equivalent to the system of three equations
y 2z
1/2
xz
2x + y
5/2 .
(1.52)
Each of the equations in (1.52) is the equation of a plane. Defining

a1 := (0, 1, 2)
a2 := (1, 0, 1)
and a3 := (2, 1, 0) ,
and defining
d1 := 1/2
d2 := 1
and d3 := 5/2 ,
The system of equations (1.52) becomes

a1 x
= d1
a2 x
= d2
a3 x
= d3 .
(1.53)
Thus, geometrically, writing the equation for a line in the form a x = d expresses the line as
the intersection of three planes.
Here is a plot showing these three planes intersecting in the line parameterized by x0 + ta:
1.2.
39
A line in R3 can always be represented as the intersection of only two planes, as illustrated below:
In (1.52), which represents a line as the intersection of three planes, each pair of the three planes
intersect in the same line. Here is is a plot of the planes one pair at a time:
Each pair of the three planes intersects in the same line.

In fact, there are infinitely many ways to represent any particular line as the intersection of two
planes: Given a line represented as the intersection of two planes, one can rotate either plane about
the line, and as long as the two planes are kept distinct, the intersection of the two planes is still the
same line.
Thus, in the system (1.52), any two of the equations suffice to specify the line, and the third
equation only provides redundant information. In general, the single vector equation a (x x0 ) = 0
corresponds to the system of scalar equations obtained by setting each components of a (x x0 )
equal to zero:
ej [a (x x0 )] = 0
for
j = 1, 2, 3 .
(1.54)
But by the triple product formula,

ej [a (x x0 )] = [ej a] (x x0 ) ,
so that if we define aj = ej a, j = 1, 2, 3, the system (1.54) is equivalent to the system
aj (x x0 ) = 0
for
j = 1, 2, 3 .
(1.55)
40
But writing a = (a1 , a2 , a3 ), we have

3
X
aj aj =
j=1
3
X
3
X
aj (ej a) =
aj ej a = a a = 0 .
j=1
j=1
Therefore, if for example a1 6= 0, we have

a1 =
1
(a2 + a3 )
a1
and hence if a2 (x x0 ) = 0 and a3 (x x0 ) = 0 then automatically we have a1 (x x0 ) = 0,

showing that the first equation in the system (1.55) is redundant. Since at least one entry of a is
non-zero, there is alway one redundant equation in the system (1.55).
Because of this, it is often advantageous instead to specify lines as the solution set of a system
of two equations, each of which describes a plane. Since every line in R3 is the intersection of two
planes, we can always specify a line as the solution set of a system of two equations specifying planes:
a1 x
d1
a2 x
d2 .
(1.56)
For example, if a1 = (1, 2, 3) and a2 = (3, 2, 1) and d1 = 1 and d2 = 1, then the system of
equations (1.56) becomes
x + 2y + 3z
3x + 2y + z
= 1
(1.57)
Its solution set is the line formed by the intersection of the two planes given by
x + 2y + 3z = 1
and
3x + 2y + z = 1 .
(1.58)
Since there are infinitely many pairs of planes one can use to specify the line, there are infinitely
many systems of equations that have the line as their solution set. There is an important caveat to
keep in mind, though: Not every pair of equations of the from (1.56) defines a line. If the planes
are parallel, then either they do not intersect at all, or they are the same plane In the first case, the
intersection is the empty set, and in the second case it is a plane. But it is easy to identify when the
planes specified in ( (1.56)) are parallel: This is when a1 is a multiple of a2 which is the case if and
only if a1 a2 = 0.
Example 15 (Parameterizing a line given by a pair of equations). Consider the system of equations
x + 2y + 3z
3x + 2y + z
All three variables are present in these equations, so we can start by eliminating any one of them,
using either equation. Let us eliminate x using the first equation:
x = 2 2y 3z .
(1.59)
1.2.
41
Substituting this into the second equation, it becomes

3(2 2y 3z) + 2y + z = 4
which yields
y=
1
2z .
2
(1.60)
This expresses y as a function of z. Now going back to (1.59), we may use (1.60) to eliminate y,
and thus to express x as a functions of z alone. We obtain
x=1+z .
Thus, x = (x, y, z) belongs to the line if and only if
x = (1 + z, 1/2 2z, z) = (1, 1/2, 0) + z(1, 2, 1) .
Definng x0 = (1, 1/2, 0) and v2 = (1, 2, 1), x(t) := x0 + tv is a parameterization of the line.
Notice that it takes two parameters to parameterize a plane in R3 , and one parameter to parameterize a line in R3 , as you might expect. Every line in R3 has a parameterization of the form
x(t) = x0 + tv ,
just as in the example we worked out above.
As with planes, there are infinitely many ways to parameterize a given line with this scheme:
Any point on the line can serve a the base point x0 , and any non-zero vector that runs parallel to
the line can serve as the direction vector.
There are also other ways of specifying planes and lines in R3 that are more geometric and less
algebraic. For example, there is a unique plane passing through any three points that do not lie on
a common line in R3 , and there there is a unique line passing through any two points in R3 . This
brings us to the following questions:
Given three non collinear points p! , p2 and p3 in R3 , how do we find the equation of the unique
plane in R3 passing through these points?
Given two distinct points z1 and z2 in R3 , how do we find a system of equations for the unique line
passing through these points?
The next examples answer these questions, and explain why we might want to know the answers.
Example 16 (When do four points belong to one plane?). Consider the points
p1 = (1, 2, 3)
p2 = (3, 2, 1)
p3 = (1, 3, 2)
and
p4 = (4, 1, 3) .
(1.61)
Do all of these points lie in the same plane?

It is easy to answer this question once we know the equation for the plane determined by the first
three points: Simply plug the fourth point into this equation. If the equation is satisfied, the point is
on the plane, and otherwise it is not.
To find the equation, choose p1 as the base point, and define
x0 := (1, 2, 3)
v1 := p2 x0 = (2, 0, 2)
and
v2 := p3 x0 = (0, 1, 1) .
42

Writing the equation of the plane in the form a (x x0 ) = 0, and plugging in p2 = x0 + v1 and
p3 = x0 + v2 , we have
a v1 = 0
and
a v2 = 0 .
Thus a must be orthogonal to v1 and v2 . We get such a vector by taking the cross product of v1 and
v2 . Thus we define
a := v1 v2 = (2, 0, 2) (0, 1, 1) = (2, 2, 2) .
We then compute a x = 2x + 2y + 2z and a x0 = 12 so the equation a (x x0 ) = 0 written
out in terms of x, y and z is 2x + 2y + 2z = 12, or, what is the same thing,
x+y+z =6 .
(1.62)
This is the equation for the plane passing through the first three points in the list (1.61). You
should check that these points do satisfy the equation.
We can now easily decide whether p4 lies in the same plane as the first three points. With x = 4,
y = 1 and z = 3, the equation (1.62) is satisfied, so it is in the plane.
Example 17 (A system of equations for the line through two points). Let x0 = (1, 1, 1) and let
x1 = (3, 2, 1). Let us find a system of equations
a1 x
d1
a2 x
d2
(1.63)
that has this line as its solution set.

To do this, we first find a parameterization x(t) = x0 + tv of the line taking the base point to be
x0 = (1, 1, 1). Since x1 is also on the line, the vector v := x1 x0 = (2, 1, 2) points along the line,
and we can choose it to be the direction vector in our parametrization.
Now consider some plane containing the line through x0 and x1 . We can use any point in the
plane, and therefore any point in the line as the base point. Hence the equation of the line can be
written as a (x x0 ) = 0, But since x1 is also in the plane
a (x1 x0 ) = 0 .
Thus, a must be orthogonal to v, and then the equation of the plane is a x a x0 .
Thus, to find our system of equations, what we need to do is to find two non-zero vecotrs a1 and
a2 that are not multiples of one another and such that
a1 v = a2 v = 0 .
Here is one way to do this. Given any vector v = (a, b, c), if b = c = 0, we can take a1 = (0, 0, 1).
Otherwise, we can take a1 = (0, c, b). Either way, a1 is a non-zero vector orthogonal to v.
Now define a2 := a1 v. By Theorem 8 and (1.39), a2 is a non-zero vecotr that is orthogonal
to both v and a1 . Evidently then, a2 is not a multiple of a1 . Let us now do the numbers:
Since v = (2, 1, 2), we take a1 = (0, 2, 1). We then compute
a2 = a1 v = (0, 2, 1) (2, 1, 2) = ( 5, 2, 4) .
1.2.
43
Next,
d1 = a1 x0 = (0, 2, 1) (1, 1, 1) = 3
and
d2 = a2 x0 = ( 5, 2, 4) (1, 1, 1) = 7 .
Thus, (1.63) can be taken to be
2y + z
5x + 2y + 4z
= 3
= 7 .
We can now check our work: computing, we find that x0 and x1 both satisfy this system of
equations.
Example 18 (Finding an orthonormal parameterization of a plane). Consider the plane given by
2x + y + 2z = 1 .
We will now find a parameterization of this plane of the form
x(s, t) = x0 + su1 + tu2
where {u1 , u2 } is orthonormal. As we shall see in the next subsection, such parameterizations are
especially convenient for many computations.
First, to find a base point, we need one solution of the equation. Choosing x = z = 0, the
equation reduces to y = 1, and so x0 = (0, 1, 0) lies on the plane, and we choose it as our base point.
The normal vector is a = (2, 1, 2), and so u1 and u2 must be orthonormal vectors that are each
orthogonal to a. In Example 10 we found such vectors when we found a right handed orthonormal
basis {u1 , u2 , u3 } in which
u3 =
1
1
(2, 1, 2) =
a.
3
kak
Thus, the parameterization we seek is given by

1
1
x(s, t) = (0, 1, 0) + s (1, 2, 0) + t (4, 2, 5) .
5
3 5
This may seem like a cumbersome sort of parameterization, due to the square roots. however,
we shall see the advantages of orthonormality in the next subsection.
1.2.4
Distance problems
Consider a point p R3 and a line parameterized by x(t) = x0 + tv. Which point on the line comes
closest to p? To answer this question, we seek to find the value of t that minimizes the distance
between x(t) and p. Actually, we can avoid a square root if instead we seek to minimize the square
of the distance, and this will produce the same value of t. (Think about why this is.)
Therefore, we seek to minimize the function
f (t) := kp x0 tvk2 = (p x0 tv) (p x0 tv) = kp x0 k2 2t(p x0 ) v + t2 kvk2 .
44
Thus, f (t) has the form a 2bt + ct2 for some numbers a, b and c. Since c > 0, the graph is an
upwards parabola, and there is a unique value t0 such that f (t0 ) < f (t) for all t 6= t0 .
To find it, we differentiate and solve 0 = f 0 (t0 ) = 2b + 2ct0 , which yields
t0 =
p x0 ) v
b
.
=
v
kvk2
Thus, the point on the line that is closest to p is

q := x0 +
(p x0 ) v
v,
kvk2
p x0 ) v
v = (p x0 ) where (p x0 ) is the orthogonal component of
kvk2
p x0 with respect to v. Thus, the distance is k(p x0 ) k. This is intuitively natural: To move
and p q = (p x0 )
from p directly to the line, one should move in a direction that is orthogonal to v. We have proved
the following result:
Theorem 11 (Closest point to a line). Consider any line in R3 parameterized by x(t) = x0 + tv.
Let p be any point in R3 . Then there is a unique point q in the line such that for all other points x
in the line,
kp qk < kp xk .
That is, the distance from p to q is less than the distance from p to any other point in the line.
Moreover, we have

kp qk =
(p x0 )

1
= k(p x0 ) k ,
((p
x
)
v)v
0

2
kvk
and
q := x0 +
(1.64)
1
((p x0 ) v)v .
kvk2
where (p x0 ) is the orthogonal component of p x0 with respect to v.

Definition 17 (Distance from a point to a line). Given any line in R3 and any point p R3 , the
distance from p to the line is defined to be kp qk where q is the unique point on the line that comes
closest to p.
If we consider the analogous problem for a plane, we will be led to try and find minimizing values
of a function
f (s, t) = kp (x0 + sv1 + tv2 )k2 .
Finding minima and maxima of functions of several variables is a subject we shall study later in this
course, so it might appear that we would be getting ahead of ourselves by discussing this problem
now. However, it turns out that by making a judicious coice of coordinates we can set the problem
up so that it can be solved by inspection. This works in many problems. Let us first use this
approach to give an alternative solution to the problem of finding the distance from a point to a line.
Let us choose an orthonormal basis {u1 , u2 , u3 } that is well-adapted to the geometry of our line.
We have seen in Example 10 that we can always construct such an orthonormal basis of R3 with
1
u3 =
v.
kvk
1.2.
45
Theorem 5 tells us that the length of any vector w can be computed in terms of this basis through
kwk2 = (w u1 )2 + (w u2 )2 + (w u3 )2 .
Applying this with w = p x(t) = p x0 tv, we and remembering that u1 and u2 are orthogonal
to v, we have
kp x(t)k2 = ((p x0 ) u1 )2 + ((p x0 ) u2 )2 + ((p x0 tv) u3 )2 .
Notice that t only enters in the third term of this sum of squares. Therefore, we can make this sum
of squares as small as possible by choosing t = t0 so that the third term is zero. This requires
(p x0 t0 v) v = 0
so that
t0 =
(p x0 ) v
,
kvk2
which is what we found before by differentiating. From here onwards, the rest of the analysis is the
same.
Example 19. Consider the line parameterized by x0 + tv with x0 = (2, 2, 3) and v = (1, 2, 1). Let
p = (1, 2, 3). What is the distance from p to this line? We obtain u3 by normalizing v, and then we
apply (1.64), finding

1
k( 1, 4, 0)k [( 1, 4, 0) (1, 2, 1)]2
6
2
1/2

=
49
17
6
1/2
.
We now turn to the problem of finding the distance between a point and a plane. We know how
to find a parameterization of a plane given an equation for it, so we may as well specify the plane by
an equation. We will prove:
Theorem 12 (Closest point to a plane). Consider any plane in R3 given by an equation of the form
a (x x0 ) = 0. Let p be any point in R3 . Then there is a unique point q in the plane such that for
all other points x in the plane,
kp qk < kp xk .
That is the distance from p to q is less than the distance from p to any other point in the plane.
Moreover, we have
kp qk =
1
|a (p x0 )| ,
kak
q = x0 + (p x0 ) = p
1
((p x0 ) a)a .
kak2
(1.65)
(1.66)
where (p x0 ) is the orthogonal component of p x0 with respect to a.

Proof: This is easy if one uses an orthonormal basis that is adapted to the geometry of the plane.
As we have seen in Example 10, it is always possible to find an orthonormal basis {u1 , u2 , u3 } of R3
such that u3 is the unit vector in the direction of a.
Then, as seen in Example 18, the plane is parameterized by x(s, t) = x0 + su1 + tu2 . By
Theorem 5, the square of the Euclidean distance from p to the general point x(s, t) in the plane is
46
given by
kp x(s, t)k2
= k(p x0 ) su2 tu3 k2
=
[(p x0 su2 tu3 ) u1 ]2 + [(p x0 su1 tu2 ) u2 ]2 + [(p x0 su1 tu2 ) u3 ]2 .
[(p x0 ) u1 s]2 + [(p x0 ) u2 t]2 + [(p x0 ) u3 ]2 .
The result is a sum of three squares. The variable s enters only in the first term, and the variable
t enters only in the second. We make the sum of the three terms as small as possible by choosing s
1
and t to make the first and second terms zero. Thus, since u3 =
a,
kak
kp x(s, t)k2 [(p x0 ) u3 ]2 =
1
|(p x0 ) a| ,
kak
which proves (1.65), and there is equality if and only if s = (p x0 ) u1 and t = (p x0 ) u2 . Thus,
the closest point q is given by x(s, t) for these values of s and t:
q := x0 + [(p x0 ) u1 ]u1 + [(p x0 ) u2 ]u2 .
(1.67)
Then since, by Theorem 5,

p x0 = [(p x0 ) u1 ]u1 + [(p x0 ) u2 ]u2 + [(p x0 ) u3 ]u3 ,
and
[(p x0 ) u1 ]u1 + [(p x0 ) u2 ]u2 = (p x0 )
1
((p x0 ) a)a .
kak2
we can rewrite (1.67) as

q=p
1
((p x0 ) a)a ,
kak2
and then since

(p x0 ) = (p x0 )
1
((p x0 ) a)a ,
kak2
this proves (1.66).

Definition 18 (Distance from a point to a plane). We define the distance from a point p R3 to a
plane in R3 to be the distance between p and the point q in the plane that comes closest to p.
Theorem 12 shows the distance between the plane given by a (x x0 ) = 0 is
1
|a (p x0 )|.
kak
Note that this is zero if and only if p belongs to the plane.

Example 20. Let p = (1, 1, 1) and consider the plane given by x + 2y + 3z = 4. What is the
distance between p and the plane?
To solve this problem ,we would first like to write the equation for the plane in the form a (x
x0 ) = 0. To find a base point x0 , let us look for a solution of x + 2y + 3z = 4 with y = z = 0. Then
we must have x = 4, and so x0 = (4, 0, 0)).
We can write x + 2y + 3z = 4 in the form a x = d by defining a = (1, 2, 3) and d = 4, and thus
we may take a = (1, 2, 3) as the normal vector.
1.2.
47
We then apply (1.65), to find that the distance is:

1
1
2
|(1, 1, 1) (4, 0, 0)) (1, 2, 3)| = |( 3, 1, 1)) (1, 2, 3)| = .
14
14
14
To find the point q in the plane that is closest to p, we use the second formula in (1.66):
q = (1, 1, 1)
1
1
(( 3, 1, 1)) (1, 2, 3))(1, 2, 3) = (8, 5, 10) .
14
7
You can check the computations by verifying that this point does satisfy x + 2y + 3z = 4.
We will close this subsection by considering another distance problem that involves minimizing a
function of two variables: Finding the distance between two lines. Again, if we work with a judiciously
chosen system of coordinates, the minimization problem can be solved by inspection.
Consider two lines, `1 and `2 , parameterized by x1 (s) = x1 + sv1 and x2 (t) = x2 + tv2 . Let
us assume that v1 and v2 are not multiples of one another, so that the lines are not parallel. (The
parallel case is easier; we shall come back to it.) What values of s and t minimize kx1 (s) x2 (t)k?
To answer this, let {u1 , u2 , u3 } be an orthonormal basis of R3 in which u1 is orthogonal to both
v1 and v2 , and in which u2 is orthogonal to v1 (and of course to u1 ). To produce this basis, define
u1 =
1
v1 v2
kv1 v2 k
and then
u2 =
1
v1 u1 ,
kv1 k
By the properties of the cross product, u1 is a unit vector orthogonal to both v1 and v2 , and u2 is
a unit vector orthogonal to both u1 and v1 . Finally, we define u3 = u1 u2 , and this gives us the
orthonormal basis we seek. Since v1 is orthogonal to both u1 and u2 , u3 must be a non-zero multiple
of v1 .
We compute kx1 (s) x2 (t)k2 in terms of coordinate for this basis. To simplify the notation,
define b by b := x1 x2 . Then
kx1 (s) x2 (t)k2
= kb + sv1 tv2 k2
=
[(b + sv1 tv2 ) u1 ]2 + [(b + sv1 tv2 ) u2 )]2 + [(b + sv1 tv2 ) u3 )]2
[b u1 ]2 + [b u2 t(v2 u2 )]2 + [b u3 + s(v1 u3 ) t(v2 u3 )]2
This is a sum of three squares. The first does not depend on s or t. The second depends only
on t, and we can make it zero by choosing
t = t0 :=
b u2
v2 u2
(1.68)
With this choice of t, we can then make the third term zero by choosing
s = s0 :=
t0 (v2 u3 ) b u3
.
v1 u3
(1.69)
This then leaves only the first term, which is then the square of the minimal distances. However,
we have to first verify that we are not dividing by zero in (1.68) and (1.69).
First, since v2 is orthogonal to u1 ,
v2 = (v2 u2 )u2 + (v2 u3 )u3 .
48
If v2 u2 were zero, then v2 would be a multiple of u3 , and hence of v1 , since u3 is a multiple of

u1 . However, since we are assuming that the lines are not parallel, v2 is not a multiple of v1 , and so
v2 u2 6= 0. This shows that t0 is well defined.
Even more simply, since u3 is a non-zero multiple of v1 , v1 u3 6= 0, which shows that s0 is well
defined.
Thus, for any choices of s and t,
kx1 (s) x2 (t)k kx1 (s0 ) x2 (t0 )k = |b u1 | ,
(1.70)
and there is equality on the left if and only if s = s0 and t = t0 . Thus, (1.70) gives the distance
between the two lines. We then define the distance between the two lines to be the distance between
these two closest points.
Example 21 (The distance between two lines in R3 ). Consider the two lines parameterized by
(1, 2, 3) + s(1, 4, 5)
and
(2, 1, 1) + t( 2, 1, 2) .
To calculate the distance between these two lines, we first need to compute a unit vector u1 that is
orthogonal to both (1, 4, 5) and ( 2, 1, 2). Computing the cross product, we find
(1, 4, 5) ( 2, 1, 2) = (13, 12, 7) .
Then since k(13, 12, 7)k =
132 + 122 + 72 =
u1 =
362,
1
(13, 12, 7) .
362
Next, we compute b in (1.70), which is the difference of the base points:

b = (1, 2, 3) (2, 1, 1) = ( 1, 3, 2) .
Finally we compute
( 1, 3, 2) (13, 12, 7)
13 36 + 14
35
=
=
.
362
362
362
Thus, the distance between he two lines is 35/ 362. If we had wanted to find the point on the first
b u1 =
line that comes closes to the second, and the point of the second line that comes closest to the first,
we would compute the rest of the orthonormal basis. But to compute the distance, we need only u1 .
Now, what about the case of parallel lines? It is left to the reader to show that if the lines are
parallel, the distance from any point on the first line to the second line is independent of the choice
of the point on the first line. Thus, this problem reduces to the problem of computing the distance
between a point and a line.
Altogether, we have proved:
Theorem 13 (The distance between two lines). The distance between two non-parallel lines in R3
parameterized by x1 (s) = x0 + sv1 and x2 (s) = x0 + tv2 is
|(x1 x2 ) (v1 v2 )|
.
kv1 v2 k
1.3. SUBSPACES OF RN
49
If the two lines are parallel, so that v1 v2 = 0, the distance is the distance from x1 to the second
line; i.e.,
k(x1 x2 ) k
where (x1 x2 ) is the component of x1 x2 orthogonal to v1 , or, what is the same thing, orthogonal
to v2 .
Notice that each of the three distance problems solved in this subsection involved minimizing a
squared distance over the certain parameters, and that in each case, this minimization problem was
rendered transparent by an appropriate choice of an orthonormal basis.
1.3
1.3.1
Subspaces of Rn
Dimension
In this section we study the generalization to higher dimensions of lines and planes through the origin
in R3 . The results proved here will be very useful to us in our study of multivariable calculus.
Definition 19 (Subspaces of Rn ). A non-empty subset V Rn is a subspace of Rn in case for every
x, y V and every pair of numbers a, b,
ax + by V .
(1.71)
Note that the condition in the definition is satisfied in a trivial way in case either V = Rn or
V = {0}. These are the trivial subspaces of Rn . By a non-zero subspace of Rn , we mean a subspace
that is not simply {0}.
Taking a = b = 0, we see that 0 belongs to every subspace V Rn . For n = 3, lines and
planes through the origin are subspaces: Consider a line through the origin with the parametric
representation x(t) = tv. Then
a(t1 v) + b(t2 v) = (at1 + bt2 )v ,
so that (1.71) is satisfied. The same reasoning using the parametric representation x(s, t) = su + tv
applies to planes through the origin.
Definition 20 (Orthonormal basis of a subspace). Let V be a non-zero subspace of Rn . An orthonormal basis of V is an orthonormal set {u1 , . . . , uk } of vectors in V such that there does not
exist any non-zero vector in V that is orthogonal each vector in {u1 , . . . , uk }.
In other words and orthonormal basis of V is a maximal orthonormal set in V ; i.e., one that is
not a subset of any larger orthonormal subset of V .
Lemma 4. Every non-zero subspace V of Rn contains at least one orthonormal basis.
Proof: Let v1 be any non-zero vector in V . Define u1 := (1/kv1 k)v1 . Since V is a subspace, u1 V .
If there is no non-zero vector in in V that is orthogonal to u1 , then {u1 } is an orthonormal basis,
and we are done.
50

Otherwise, let v2 be a non-zero vector in V that is orthogonal to u1 , and define u2 := (1/kv2 k)v2 .
Since V is a subspace, u2 V . Thus, {u1 , u2 } is an orthonormal set in V. If there is no non-zero

vector in in V that is orthogonal to u1 and u2 , then {u1 , u2 } is an orthonormal basis.
Otherwise, we can add another vector. This process must stop at most when we have n vectors
in the set, since V Rn , and by Lemma 1, there does not exist any set of n + 1 orthonormal vectors
in Rn .
Lemma 5. Let V be a non-zero subspace of Rn , and let {u1 , . . . , uk } be an orthonormal basis of V .
Then for each v V ,
v=
k
X
(v uj )uj .
(1.72)
j=1
Moreover, all orthonormal bases of V consist of the same number k of unit vectors.
Proof: Let v V , and define z := v
k
X
(v ui )ui . Since V is a subspace, z V . We compute
i=1
z uj =
k
X
!
(v ui )ui
uj = v uj v uj = 0 ,
i=1
where we have used the orthonormality of {u1 , . . . , uk }.

If z 6= 0, there is a non-zero vector in V orthogonal to each vector in {u1 , . . . , uk }. This is not
possible since {u1 , . . . , uk } is an orthonormal basis. Hence z = 0, and (1.72) is proved.
Next, Consider the function c from V to Rk given by
c(v) := (v u1 , . . . , v uk ) .
We now claim that for any v, w V , v w = c(v) c(w). Indeed, by the first part:
! k
k
X
X
vw =
(v ui )ui (v uj )uj
i=1
k
X
j=1
(v ui )(v uj )ui uj
i,j=1
k
X
(v ui )(v ui ) = c(v) c(w) .
i,=1
Now suppose that {w1 , . . . , w` } is another orthonormal basis of V . Then for each i, j = 1 . . . , `,
1 i = i
c(wi ) c(wj ) = wi wj =
.
0 i 6= j
Thus {c(w1 ), . . . , c(w` )} is an orthonormal set in Rk . By Lemma 1, ` k. Since our original
orthonormal basis was arbitrary, we have seen that no orthonormal basis of V can contain more
vectors than any other. Hence, they all contain the same number of vectors.
Definition 21 (Dimension). Let V be a non-zero subspace of Rn . We define the dimension of V ,
dim(V ), to be the number of vectors in any orthonormal basis of V . By convention, we define the
dimension of the zero subspace to be zero.
1.3.2
51
Orthogonal complements
Definition 22 (Orthogonal complement). Let S be any subset of Rn , finite or infinite. The orthogonal complement of S, S , is the subset of vectors x Rn that are orthogonal to each v S.
Note that if S1 S2 , then a vector y has to satsify more conditions of the form x y = 0 to
belong to S2 than to S1 . Hence:
S1 S2
S2 S1 .
(1.73)
Example 22 (Orthogonal complements). Let {a} be a non-zero vector in R3 , and let S = {a}. Then
S is the set of all vectors orthogonal to a, i.e., the set of all x R3 such that
ax=0 .
This is the equation of the plane through the origin that is normal to a.
Likewise, let S = {a1 , a2 } be a set of two non-zero vectors in R3 that are not multiples of one
another. Then S is the line specified by
a1 x
a2 x
0.
In both of these examples, S is a subspace. This is always the case:

Theorem 14. Let S be any subset of Rn . Then S is a subspace. For any subspace V of Rn ,
dim(V ) + dim(V ) = n
(1.74)
(V ) = V .
(1.75)
and
Finally, with k := dim(V ), there exists an orthonormal basis {u1 , . . . , uk , uk+1 , . . . , un } of Rn such
that {u1 , . . . , uk } is an orthonormal basis of V , and {uk+1 , . . . , un } is an orthonormal basis of V .
Proof: Let x, y S , and let a, b be any two numbers. For any v S,
(ax + by) v = a(x v) + b(y v) = a0 + b0 = 0 .
Thus ax + by S . This shows that S is a subspace.
Now let V be a non-trivial subspace of Rn , and let {u1 , . . . , uk } be an orthonormal basis of V ,
so that dim(V ) = k. Since V is non-trivial, 0 < k < n.
While {u1 , . . . , uk } is an orthonormal basis of V , it is not an orthonormal basis of Rn itself.
Thus, as in the proof of Lemma 4, we can continue adding unit vectors to the set until we arrive at
an orthonormal basis of {u1 , . . . , un } of Rn , necessarily consisting of n vectors, and each vector in
{uk+1 , . . . , un } is orthogonal to each vector in {u1 , . . . , uk }, and hence to each vector in V .
n
X
Now, every vector x Rn can be written as x =
(x ui )ui . Every vector x V is orthogonal
i=1
to each vector in V , and thus to each vector in {u1 , . . . , uk }. Hence,

xV
x=
n
X
(x ui )ui .
i=k+1
52
Thus, x is a linear combination of the vectors in {uk+1 , . . . , un }, and cannot be orthogonal to all of
them unless x = 0, and consequently, {uk+1 , . . . , un }, is anl orthonormal basis of V . It follows that
dim(V ) = n k. This proves (1.74).
Next by (1.74) applied with V in place of V ,
dim(V ) + dim((V ) ) = n = dim(V ) + dim(V ) .
It follows that
dim((V ) ) = dim(V ) .
Also since by definition, every vector in V is orthogonal to every vector in V , every vector in V
belongs to (V ) . That is,
V (V ) .
Now let {u1 , . . . , uk } be an orthonormal basis of V . Then {u1 , . . . , uk } is an orthonormal subset of
(V ) , and since dim(V ) ) = k, no more vectors can be added to it, and it is an orthonormal
basis of (V ) . By Lemma 5 again, every vector in (V ) is a linear combination of the vectors
in {u1 , . . . , un }, and hence belongs to V . Thus V (V ) . Altogether, V = (V ) . The final
statement is now evident.
Definition 23 (Span of a set of vectors). Let S Rn . Then the span of S, Span(S), is the
k
X
set of all linear combinations of finite subsets of S; i.e., of all vectors of the form
xj xj with
j=1
{x1 , . . . , xk } S.
Since a linear combination of two linear combinations of vectors in S Rn is a again a linear
combination of vectors in S, Span(S) is a subspace of Rn . In fact:
Theorem 15 (Span and orthogonal complements). Let S be any subset of Rn . Then (S ) =
Span(S).
Proof: Let W be any subspace of Rn such that S W . By (1.73), W S , and then by (1.73)
again and (1.75), (S ) (W ) = W . In particular, since S Span(S), (S ) Span(S).
On the other hand, since subspaces are closed under taking linear combinations, and since (S )
contains S, (S ) contains every finite linear combination of vectors in S; i.e., Span(S) (S ) .
Having proved both (S ) Span(S) and Span(S) (S ) , we have the equality.
The answers to most questions concerning a k dimensional subspace V of Rn can be answered
by considering the orthonormal basis {u1 , . . . , uk , uk+1 , . . . , un } of Rn such that {u1 , . . . , uk } is an
orthonormal basis of V , and {uk+1 , . . . , un } is an orthonormal basis of V that was constructed in
the previous theorem.
For example, since x Rn belongs to V is and only if uj x = 0 for each j = k + 1, . . . , n, V is
the solution set of the n k equations
uk+1 x
..
.
un x
0
.
= ..
=
53
Moreover, the function

(t1 , . . . , tk ) 7 x(t1 , . . . , tk ) :=
k
X
tj uk
j=1
is a one-to-one function from Rk onto V , and hence is a parameterization of V . The inverse function,
which we denote by c, is the corresponding coordinate map on V , and is given by
c(x) = (u1 x, . . . , uk x) .
As you can readily check,
c(x(t1 , . . . , tk )) = (t1 , . . . , tk ) .
Indeed, we call {u1 , . . . , uk } an orthonormal basis of V since it is the basis of a system of orthonormal
coordinate on V .
Next, let us consider the higher dimensional analog of the problem of finding the distance between
a point and a line or a plane in R3 . Let b be any point in Rn . Can we find a point x V that comes
closer to b than any other point in V ? Yes, using this well-adapted basis: For any x V ,
kb xk2
n
X
((b x) uj )2
j=1
n
X
(b uj x uj )2
j=1
k
X
(b uj x uj ) +
j=1
n
X
(b uj )2
j=k+1
where in the last line we have used the orthogonality of every vector in V to each uj , j > k. Evidently
we make this sum of squares as small as possible by choosing
x=
k
X
(b uj )uj
(1.76)
j=1
since this choice, and this choice alone, makes
k
X
(b uj x uj )2 = 0. The conclusion is that the
j=1
vector x defined in (1.76) comes closer to b than any other vector in V . We define the distance
between b and V to be kb xk for this choice of x, and we see from the calculation above that this
distance is
n
X
1/2
(b uj )2
j=k+1
This result generalizes our previous results on the distances between points and lines and points and
planes to arbitrarily many dimensions. Of course, to use these formulas, one has to be able to find
the well-adapted basis {u1 , . . . , uk , uk+1 , . . . , un } of Rn . We have effective means for doing this for
n = 3, and we shall develop effective means for doing this in every dimension as we go along. The
main point that we want to make here, with which we conclude this chapter, is that well-adpated
orthonormal bases provide the key to a great many geometric problem that we shall consider, no
matter how many variables are involved.
54
1.4
Exercises
1.1 Let a = (3, 1), b = (2, 1) and c = (1, 3). Express a as a linear combination of b and c. That
is, find numbers s and t so that a = sb + tc.
1.2 Let a = (5, 2), b = (2, 1) and c = (1, 1). Express each of these three vectors as a linear
combination of the other two.
1.3 Let x = (1, 4, 8) and y = (1, 2, 2). Compute the lengths of each of these vectors, and the angle
between them.
1.4 Let x = (4, 7, 4, 1, 2, 2) and y = (2, 1, 2, 2, 1, 1). Compute the lengths of each of these
vectors, and the angle between them.
1.5 Let x = (4, 7, 4) and y = (2, 1, 2). Compute the lengths of each of these vectors, and the angle
between them.
1.6 Let x = ( 5, 2, 5) and y = (1, 2, 1). Is the angle between x and y acute or obtuse? Justify
your answer.
1.7 Let
u1 =
1
(1, 4, 8)
9
u2 =
1
(8, 4, 1)
9
and u3 =
1
(4, 7, 4) .
9
(a) Show that {u1 , u2 , u3 } is an orthonormal basis of R3 . Is it a right-handed orthonormal basis?

Justify your answer.
(b) Find numbers y1 , y2 and y3 such that
y1 u1 + y2 u2 + y3 u3 = (10, 11, 11) .
What are the lengths of the vectors (10, 11, 11) and (y1 , y2 , y3 )? give calculations or an explanation
in each case.
1.8 Let a, b and c be any three vectors in R3 with a 6= 0. Show that b = c if and only if
ab=ac
and a b = a c .
1.9 Let a = (1, 1, 1)

(a) Find a vector x such that
x a = ( 7, 2, 5)
and
xa=0 .
(b) There is no vector x such that

x a = (1, 0, 0)
and
xa=0 .
Show that no such vector exists.

1.10 (a) Let a = ( 1, 1, 2) and b = (2, 1, 1). Find all vectors x, if any exist, such that
a x = ( 2, 4, 3)
and
bx=2 .
1.4. EXERCISES
55
If none exist, explain why this is the case.

(b) Let a = ( 1, 1, 2) and b = (2, 1, 1). Find all vectors x, if any exist, such that
a x = (2, 4, 3)
bx=2 .
and
If none exist, explain why this is the case.

(c) Among all vectors x such that ( 1, 1, 2) x = ( 2, 4, 3), find the one that is closest to
(1, 1, 1).
1.11 (a) Let a and b be orthogonal vectors. Define a sequence of vectors {bn } by
bn+1 = a bn
and
b0 = b .
Show that for all positive integers m

b2m+2 = (1)m kak2m b .
How do you have to adjust the formula if the hypothesis that a and b are orthogonal is dropped?
1
(b) Let a = (2, 1, 2) and b = (1, 1, 1). With bn defined as in part (a), compute b99 .
3
1.12 (a) Let a, b and c be three non-zero vectors in R3 . Define a transformation f from R3 to R3 by
f (x) = a (b (c x))) .
Show that f (x) = 0 for all x R3 if and only if b is orthogonal to c, and a is a multiple of c.
1.13 (a) Let a, b and c be three non-zero vectors in R3 . Show that
|a (b c)| kakkbkkck

1
1
1
and there is equality if and only if
a,
b,
c is orthonormal.
kak kbk kck
1.14 Let P1 the plane through the three points a1 = (1, 2, 1) a2 = ( 1, 2, 3) and a3 = (2, 3, 2).
Let P2 denote the plane through the three points b1 = (1, 1, 0) b2 = (1, 0, 1) and b3 = (0, 1, 1).
(a) Find equations for the planes P1 and P2 .
(b) Parameterize the line given by P1 P2 , and find the distance between this line and the point a1 .
(c) Consider the line through b1 and b2 . Determine the point of intersection of this line with the
plane P1 .
1.15 Consider the vector v = (1, 4, 3). Find an orthonormal basis of R3 whose third vector is a
multiple of v.
1.16 Consider the vector a = (1, 4, 3) and b = (3, 2, 1). Find an orthonormal basis of R3 whose third
vector is orthogonal to both a and b.
1.17 Consider the plane passing through the three points
p1 = ( 2, 0, 2)
p2 = (1, 2, 2)
and
p3 = (3, 1, 2)
56
and the line passing through

z0 = (1, 4, 2)
z1 = (0, 3, 1)
and
(a) Find a parametric representation x(s, t) = x0 + sv1 + tv2 for the plane.
(b) Find a parametric representation z(u) = z0 + uw for the line.
(c) Find an equation for the plane.
(d) Find a system of equations for the line.
(e) Find the points, if any, where the line intersects the plane.
(f ) Find the distance from p1 to the line.
(g) Find the distance from z0 to the plane.
1.18 Consider the plane passing through the three points
p1 = ( 1, 3, 0)
p2 = (5, 1, 2)
and
p3 = (0, 3, 4)
and the line passing through

z0 = (1, 1, 1)
z1 = (1, 2, 2)
and
(a) Find a parametric representation x(s, t) = x0 + sv1 + tv2 for the plane.
(b) Find a parametric representation z(u) = z0 + uw for the line.
(c) Find an equation for the plane.
(d) Find a system of equations for the line.
(e) Find the points, if any, where the line intersects the plane.
(f ) Find the distance from p1 to the line.
(g) Find the distance from z0 to the plane.
1.19 Consider the two lines parameterized by
(1, 1, 0) + t(1, 1, 2)
and
(2, 0, 2) + s( 1, 1, 0) .
(a) These lines intersect. Find the point of intersection.

(b) Find an equation for the plane P containing these two lines.
1.20 Consider the plane given by
2x y + 3z = 4 .
Let p = ( 1, 3, 0). What is the distance from p to the plane?
1.21 Consider the plane given by
x 3y + z = 2 .
Let p = ( 2, 5, 1). What is the distance from p to the plane?
1.4. EXERCISES
57
1.22 Consider the line ` given by

2x y + 3z
x+y+z
=4
=2.
Find a parametric representation of the line obtained by reflecting this line through the plane x +
3y z = 1. That is; the outgoing line should have as its base point the intersection of the incoming
line and the plane, and its direction vector should be hu (v) where v is the incoming direction vector,
and u is a unit normal vector to the plane.
1.23 Consider the line ` given by
x 3y + z
2y + z
3.
Find a parametric representation of the line obtained by reflecting this line through the plane x +
2y z = 1. Find a parametric representation of the line obtained by reflecting this line through the
plane x + 3y z = 1. (See the previous exercise.)
1.24 Let x = (5, 2, 4, 2). Let u be a unit vector such that hu (x) is a multiple of e1 . What are the
possible values of this multiple? Find four such unit vectors u.
1.25 Consider two lines in R3 given parametrically by x1 (s) = x1 + sv1 and x2 (t) = x2 + tv2 where
x1 = (1, 2, 1)
x2 = (1, 1, 0)
v1 = (1, 0, 1)
and
v2 = (2, 1, 1) .
Compute the distance between these two lines.

x1 = (1, 2, 3)
x2 = (2, 0, 2)
v1 = (1, 2, 2)
v2 = ( 2, 1, 1) .
and
Compute the distance between these two lines.

x2 = (1, 1, 1)
x1 = (3, 2, 1)
v1 = (3, 5, 1)
v2 = ( 1, 3, 3) .
and
Find the point on the first line that is closest to the second line, the point on the second line that is
closest to the first line, and the distance between these two lines.
x1 = (1, 2, 1)
x2 = (2, 1, 5)
v1 = (1, 4, 2)
and
v2 = (1, 1, 2).
Find the point on the first line that is closest to the second line, the point on the second line that is
closest to the first line, and the distance between these two lines.
1.29 Let {u1 , u2 , u3 } be given by
u1 =
1
(1, 2, 2)
3
u2 =
1
(2, 1, 2)
3
and
u3 =
1
(2, 2, 1) .
3
58
(a) Verify whether {u1 , u2 , u3 } is, or is not, an orthonormal basis of R3 .

(b) Find a unit vector u so that hu (u1 ) = e1 .
(c) With this came choice of u, compute hu (u2 ) and hu (u3 ).
u1 =
1
(1, 4, 8)
9
u2 =
1
(8, 4, 1)
9
and
u3 =
1
(4, 7, 4)) .
9

(c) With this came choice of u, compute hu (u2 ) and hu (u3 ).
u1 =
1
(1, 2, 2)
3
1
u2 = (0, 1, 1)
2
and
1
u3 = (4, 1, 1)) .
3 2

(c) With this same choice of u, compute hu (u2 ) and hu (u3 ).
1.32 Let V1 and V2 be two subspaces of Rn .
(a) Show that V1 V2 is a subspace of Rn .
(b) Define V1 + V2 to be the set of all vectors z Rn such that z = x + y for some x V1 and some
y V2 . Show that V1 + V2 is a subspace of Rn .
1.33 Let V1 and V2 be two subspaces of Rn . Using the results and notation from the previous exercise,
show that
dim(V1 V2 ) + dim(V1 + V2 ) = dim(V1 ) + dim(V2 ) .
1.34 For n > 3, an n 1 dimensional V subspace of Rn is called a hyperplane through the origin.
The orthogonal complement V is a one dimensional subspace. In this case, starting from the
equation of the hyperplane, it is easy to write down an orthonormal basis {u1 , . . . , un1 , un } such
that {u1 , . . . , un1 } is an orthonormal basis of V , and such that {un } is an orthonormal basis of V :
Let a be a non-zero vector in Rn , and let V be the solution set of a x = 0. Define the uni vector
w = (1/kak)a. Let u be a unit vector such that the Householder reflection hu satisfies
hu (w) = en .
Define
uj = hu (ej )
for j = 1, . . . , n .
Show that with these definitions, {u1 , . . . , un1 } is an orthonormal basis of V , and such that {un }
is an orthonormal basis of V .
1.4. EXERCISES
59
1.35 We use the notation and results of the previous exercise. Consider the hyperplane V through
the origin in R4 given by
2x + 2y 7z + 4w = 0 .
Let b = (1, 2, 0, 2). Find the point x V that is closest to b and find the distance between b and V .
1.36 Show that for all vectors a, b, c and d in R3 ,
(a b) (c d) = (a c)(b d) (a d)(b c) .
1.37 Show that for all a, b and c in R3 ,
(b c) [(c a) (a b)] = |a (b c)|2 .
60
Chapter 2
DESCRIPTION AND
PREDICTION OF MOTION
2.1
Functions from R to Rn and the description of motion
In many ways, the simplest multivariable functions are functions from R to Rn for some n 1.
These are functions that have one input variable (one independent variable), and n output variables
(n dependent variables).
If n = 2 or if n = 3, we can think of the output variable x as the coordinate vector of a point
moving in R2 or R3 , and we can think of the input variable t as the time so that the function gives
us the location of a moving point at time t. We then write the function as x(t). Such functions are
also called vector valued functions of a real variable.
Example 23. Consider the function x(t) of the real variable t with values in R3 given by
x(t) = ( cos(t) , sin(t) , 1/t) .
(2.1)
Here is a three dimensional plot of the curve traced out by x(t) as for t 1 20. (Since x3 (t) = 1/t
is a decreasing function of t, the x(0) end of the curve is at the top.)
1
0.8
0.6
0.4
0.2
1
0.5
0
0.5
1 1
0.5

61
0.5
62
CHAPTER 2. DESCRIPTION AND PREDICTION OF MOTION
2.1.1
Continuity of functions from R to Rn
Vector valued functions of one real variable that describe particle motion usually have certain regularity properties: For example, particle motions are usually at least continuous:
Definition 24 (Convergence and continuity in Rn ). A sequence of vectors {xj } in Rn converges to
z Rn in case for each > 0, there is a natural number N so that
j N
kxj zk ,
in which case we say that z is the limit of the sequence and write
lim xj = z .
A function x(t) defined on an open interval (a, b) R with values in Rn is continuous at

t0 (a, b) in case for each > 0, there is a real number > 0 so that
|t t0 |
kx(t) x(t0 )k ,
(2.2)
in which case we write limtt0 x(t) = x(t0 ). The function x(t) is said to be continuous if it is
continuous at each point in its domain. Such a function is often called a curve in Rn .
We begin with several observations. First, a sequence {xj } cannot have more than one limit,
and so our reference to the limit in the definition above does make sense. To see this, suppose that
lim xn = y
and
lim xn = z .
Fix any > 0. Then there is a natural number N so that for all j N , we have kxj yk /2 and
kxj zk /2. Hence by the triangle inequality, Theorem 3, for any such j,
ky zk ky xj k + kxj zk

+ =.
2 2
Thus, for every > 0, ky zk . This is only possible in case y = z.

Next, observe that
kx(t) x(t0 )k2 =
n
X
|xj (t) xj (t0 )|2 .
(2.3)
j=1
Hence, for any single j, |xj (t) xj (t0 )| kx(t) x(t0 )k so that, whenever (2.2) is true,
|t t0 |
|xj (t) xj (t0 )| ,
(2.4)
meaning that whenever x(t) is continuous, each of its entries is a continuous real valued function.
Conversely, suppose each of the entry functions xj (t) is continuous. at t0 . Then for each j, and
(j)
each > 0, there is a
> 0 so that
|t t0 | (j)
|xj (t) xj (t0 )| ,
Then if we define = min{(1) , . . . , (n) }}, > 0, (2.5) implies that for |t t0 | /n ,
v
uX
q
u n
2
t
|xj (t) xj (t0 )| n(/ n)2 = .
j=1
(2.5)
2.1. FUNCTIONS FROM R TO RN AND THE DESCRIPTION OF MOTION
63
Hence,
|t t0 | /n
kx(t) x(t0 )k ,
which means that x(t) is continuous at t0 . In summary:

A vector valued function x(t) of a real variable t is continuous at t0 if and only if each of its entry
functions xj (t) is a continuous at t0 .
Therefore, everything we already know about continuity for single variable functions applies
directly to vector valued functions of a real variable: Since sin(t), cos(t) and 1/t are all continuous
functions on the interval (1, 20), the vector valued function in Example 23 is continuous.
Finally, there is a close connection between convergence along a curve, as in limtt0 x(t) = x(t0 ),
an convergence of sequences, as in limj xj = z.
Theorem 16 (Convergence along curves and sequences). Let x(t) be a function from (a, b) R to
Rn . Then, for any t0 (a, b), x(t) is continuous at t0 if and only if for each sequence {tj } of numbers
in (a, b) such that
lim tj = t0 ,
it is the case that

lim x(tj ) = x(t0 ) .
(2.6)
Proof: Suppose that x(t) is continuous at t0 , and pick > 0. Since x(t) is continuous, there is a
> 0 such that (2.2) is true. Let {tj } be any sequence of numbers in (a, b) such that limj tj = t0 .
Then there is an N so that for all j N , |tj t0 | < . Hence, for all j N , kx(tj ) x(t0 )k .
Since is arbitrary, this proves (2.6).
Conversely, suppose that x(t) is not continuous at t0 . Then for some > 0, and each j > 0,
there is a tj (a, b) so that |tj t0 | 1/j, but kx(tj ) x(t0 )k > . But then (2.6) is impossible, and
so whenever x(t) is not continuous at t0 , there exists a sequence {tj } of numbers in (a, b) such that
limj tj = t0 , and for which (2.6) fails.
2.1.2
Differentiability of functions from R to Rn
Physical motions of particles are usually not only continuous, but differentiable. In fact, as we shall
explain later, as a consequence of Newtons second law, as long as no infinite forces act on a particle,
its motion will be described by a curve that is at least twice differentiable.
Before we define differentiability, let us explain the idea of the derivative:
To say that x(t) is differentiable at t = t0 means that if we look at the graph of this function, and
zoom in close enough on the graph near x(t0 ), it will look indistinguishable from some line through
x(t0 ). That is, under sufficiently high magnification every segment of the graph of a differentiable
vector valued function looks like a straight line segment.
Consider a vector valued function x(t) = (x(t), y(t), z(t)) with values in R3 . Suppose that each
of the coordinate functions x(t), y(t) and z(t) is differentiable at t = t0 in the familiar single-variable
64
sense. Then we have, for t t0 ,

x(t)
x(t0 ) + (t t0 )x0 (t)
y(t)
y(t0 ) + (t t0 )y 0 (t)
z(t)
z(t0 ) + (t t0 )z 0 (t) .
(2.7)
Define x0 = x(t0 ) and v = (x0 (t0 ), y 0 (t0 ), z 0 (t0 )). We can express these approximations in vector
notation as
x(t) x0 + (t t0 )v ,
telling us which straight line segment the graph of x(t) should look like near x0 . This line, parameterized by x0 + tv, is called the tangent line to the graph of x(t) at t = t0 .
Example 24 (A tangent line). Consider the vector valued function
x(t) = (x(t), y(t), z(t)) = (t , 23/2 t3/2 /3 , t2 /2) .
Then one computes, using single variable calculus,
x0 (t)
y 0 (t)
21/2 t1/2
z 0 (t)
t.
Taking t0 = 1 we have from (2.7) that for t t0 ,

x(t) x0 + (t t0 )v = (1, 23/2 /3, 1/2) + (t 1)(1, 21/2 , 1) .
(2.8)
Here is a graph showing both the curve x(t) and the tangent line x0 + tv for 0 t 2:
As you can see, the straight line is a very close match to the curve for t t0 = 1: Had we
zoomed in more, and shown the two graphs only for 0.9 t 1.1, the two graphs would have been
pretty much indistinguishable. On such an interval, kx(t) [x0 + (t t0 )v]k would be very small
compared to |t t0 |. That is, for some very small > 0, we would have
kx(t) [x0 + (t t0 )v]k |t t0 | ,
at least for those values of t visible in our zoomed in graph.
65
There are two important sizes displayed in the graph in the previous example:
(1) The zoom factor, or size of piece of segment that is viewed. The traditional notation for this is
; we zoom in so that the piece of the curve for valued of t with t0 t t0 + fills the viewing
space.
(2) The accuracy of fit factor. The traditional notation for this is . By construction, the line and
the curve agree at t = t0 . However, as t moves away from t0 , there will generally be deviation. The
accuracy of fit factor measures the worst case deviation, as a fraction of |t t0 |. The smaller this
is, the more accurate the fit is.
Roughly speaking, a curve is differentiable at t0 if, for any desired accuracy of fit , if we zoom
in to a small enough , the pieces of the curve and the line that are in the viewing area will fit to the
desired degree of accuracy.
Definition 25 (Derivatives of Vector Valued Functions). Let x(t) be a vector valued function of the
variable t. We say that x(t) is differentiable at t = t0 with derivative v in case for every > 0, there
is a > 0 so that
|t t0 |
kx(t) x(t0 ) (t t0 )vk |t t0 | .
(2.9)
In this case we say that v is the derivative of x(t) at t = t0 , and shall often denote it by writing
x0 (t0 ).
A vector valued function is differentiable in some interval (a, b) if it is differentiable for each t0
in (a, b), and then we shall write x0 (t) to denote the function associating to each t the derivative of
x(t) at t.
Now define h = t t0 , and note that we can rewrite (2.9) as

1

|h|
h (x(t + h) x(t0 )) v .
(2.10)
By what we have explained about convergence and limits in Rn , this is equivalent to

1
(x(t0 + h) x(t0 )) = v ,
h0 h
lim
(2.11)
and in particular, since limits are unique when they exist, there can be at most one vector v for
which (2.9) is true. (Of course (2.9) refers to a particular choice of t0 . If this is changed, v may
change too, but what is uniquely determined is the vector v at each fixed t0 .)
Moreover, when it comes to actually computing this vector v, we can make ready use of what
we know about single variable calculus: By what we have explained about convergence and limits in
Rn , (2.11) is equivalent to
lim
h0
1
(xj (t0 + h) xj (t0 )) = vj
h
for
j = 1, . . . , n .
So as far as doing the computations necessary to evaluate the derivative of a vector valued function,
we only need to differentiate the entries of x(t) one at a time.
To compute the derivative of x(t), you simply differentiate it entry by entry.
66
Example 25 (Computing the derivative of a vector valued function of t)). Let x(t) be given by (2.1).
Then for any t 6= 0,
x0 (t) = ( sin(t), cos(t), 1/t2 ) .
Because we differentiate vectors entry by entry without mixing the entries up in any way, familiar
rules for differentiating scalar valued functions hold for vector valued functions as well. In particular,
the derivative of a sum is still the sum of the derivatives, etc.:
(x(t) + y(t))0 = x0 (t) + y0 (t) .
(2.12)
Here is another example of how we can leverage our knowledge of single variable calculus:
If x(t) is differentiable at t = t0 , then it is continuous at t = t0 .
Indeed, we know that for each j, xj (t) is continuous at t = t0 provided it is differentiable at
t = t0 . But since both continuity and differentiability of vector functions x(t) of a single variable t
can both be checked at the level of the individual entry functions, this is all we need to know.
Remark: We warn the reader that things are more complicated when the input variable is a vector
variable: As we shall see, even for real valued functions of a vector variable, neither continuity nor
differentiability can be checked one variable at a time. But for vector valued functions of a single real
variable, the situation is much simpler.
We now turn to product rules. There are now several types of products to consider: product
rules for scalar-vector multiplication and product rules for both the dot and cross products.
Theorem 17 (Differentiating dot and cross products). Suppose that v(t) and w(t) are differentiable vector valued functions for t in (a, b) with values in Rn , and that both of these functions are
differentiable at t0 (a, b). Then v(t) w(t) is differentiable at t0 and

d
= v0 (t0 ) w(t0 ) + v(t0 ) w0 (t0 ) .
(v(t) w(t))
dt
t=t0
Also, if n = 3 so that the cross product is defined, v(t) w(t) is differentiable at t0 and

d
= v0 (t0 ) w(t0 ) + v(t0 ) w0 (t0 ) .
(v(t) w(t))
dt
t=t0
(2.13)
(2.14)
Proof: By definition we have

d
1
(v(t) w(t))
= lim (v(t0 + h) w(t0 + h) v(t0 ) w(t0 )) .
h0
dt
h
t=t0
(2.15)
Now we use the device of adding and subtracting that is used to prove the single variable product
rule to write
v(t0 + h) w(t0 + h) v(t0 ) w(t0 )
[v(t0 + h) v(t0 )] w(t0 + h)
v(t0 ) [w(t0 + h) w(t0 )]
(2.16)
Note that this identity is true because we have simply added v(t0 ) w(t0 + h) in the first line on the
right, and subtracted it back out in the second. The advantage is that now in each term, only one of
v and w is changing.
67
Combining (2.15) and (2.16), we have

d
(v(t) w(t))
dt
t=t0
=
+
=

v(t0 + h) v(t0 )
lim
w(t0 + h)
h0
h

w(t0 + h) w(t0 )
lim v(t0 )
h0
h
0
v (t0 ) w(t0 ) + v(t0 ) w0 (t0 )
The proof for cross products is exactly the same; simply replace each dot product with a cross
product in the lines above.
Finally there is the case of the product rule for scalar vector multiplication. If g(t) is a real
valued function defined on (a, b), and x(t) is an Rn valued function defined on (a, b), and if both are
differentiable at t0 (a, b), then

d
(g(t)x(t))
= g 0 (t0 )x(t0 ) + g(t0 )x0 (t0 ) .
dt
t=t0
(2.17)
We leave the proof of this to the reader - treat the components one at a time.
We close this subsection by returning to the notion of a tangent line, now that we have given a
precise definition of differentiability.
Definition 26 (Tangent line approximation). Suppose that x(t) is a function with values in Rn that
is defined on some open interval (a, b), and x(t) is differentiable at t = t0 with a < t0 < b. Then the
approximation
x(t) x(t0 ) + (t t0 )x0 (t0 )
is called the tangent line approximation, and the parameterized line on the right hand side is called
the tangent line to x(t) at t0 . The vector x0 (t0 ) is called the velocity vector at t = t0 .
We have already computed an explicit tangent line approximation (2.8) in Example 24. The
following theorem specifies the sense in which the tangent line, among all possible lines, gives the
best fit to the curve near t = t0 :
Theorem 18 (Best linear approximation). Let x(t) be differentiable at t = t0 . Then
1
kx(t0 + h) [y0 + hw]k = 0
h0 h
lim
(2.18)
if and only if y0 = x(t0 ) and w = x0 (t0 ).

Proof: It follows from (2.10) that when x(t) be differentiable at t = t0 , and y0 = x(t0 ) and
w = x0 (t0 ), then (2.18) is true.
Conversely, observe that by the continuity of x(t) at t = t0 ,
lim kx(t0 + h) [y0 + hw]k = kx(0) y0 k ,
h0
68
so unless y0 = x(t0 ), the limit in (2.18) would even diverge to +. Therefore, suppose that y0 =
x(t0 ). Then, by the Triangle Inequality, Theorem 3,
kw x0 (t0 )k
=
=
1
khw hx0 (t0 )k
|h|
1
k(x(t + h) x(t0 ) hx0 (t0 )) (x(t + h) y0 hw)k
|h|
1
1
kx(t + h) x(t0 ) hx0 (t0 )k +
kx(t + h) y0 hwk .
|h|
|h|
As h tends to zero, the first term in the last line goes to zero by the differentiability of x(t) at t = t0 ,
and the second term in the last line goes to zero by (2.18). Therefore, kw x0 (t0 )k = 0, and so
w = x0 (0).
2.1.3
Velocity and acceleration
Let x(t) represent the position of a point particle at time t. Then, x0 (t) is called the velocity. It is
often denoted by v(t). The velocity gives the rate of change of the position.
If the function v(t) = x0 (t) is differentiable, then v0 (t) is called the acceleration, and is often
denoted by a(t), so that a(t) = v0 (t) = x00 (t). In this case we say that x(t) is twice differentiable,
and twice continuously differentiable in case a(t) = x00 (t) is continuous. Thus, the acceleration is the
second time derivative of the position and gives the rate of change of the velocity vector.
Consider xj (t), the jth entry of x(t). This is a garden variety real valued function of a single
real variable. By Taylors Theorem, given any t0 ,
1
xj (t) = xj (t0 ) + x0j (t0 )(t t0 ) + x00j (t0 )(t t0 )2 + O((t t0 )3 ) .
2
Doing the same for the other coordinates, and combining the results in vector form, we have
(t t0 )2 00
x (t0 ) + O((t t0 )3 )
2
(t t0 )2
= x(t0 ) + (t t0 )v(t0 ) +
a(t0 ) + O((t t0 )3 ) .
2
(We have written the scalar multiples in front of the vectors, as we usually do.)
x(t)
= x(t0 ) + (t t0 )x0 (t0 ) +
(2.19)
Here you see what the velocity and acceleration tell you: If h = t t0 is small, then the leading
order approximation to x(t0 + h) is given by
x(t0 + h) x(t0 ) + hv(t0 ) +
h2
a(t0 ) ,
2
and the errors in this approximation are O(|h|3 ).

Example 26 (Velocity and acceleration). Let x(t) be given by x(t) = (t, 23/2 t3/2 /3, t2 /2) for t > 0.
Then, simply differentiating entry by entry, we find that for all t > 0,
v(t) = x0 (t) = (1, 21/2 t1/2 , t)
and
a(t) = x00 (t) = (0, 21/2 t1/2 , 1).
Taking t = t0 = 1, for example, we find the instantaneous position, velocity and acceleration at
t = t0 = 1:
x(1) = (1, 23/2 /3, 1/2)
v(1) = (1, 21/2 , 1)
and
a(1) = (0, 21/2 , 1) .
69
We now turn to an investigation of the magnitudes and directions of the velocity and the acceleration, with the goal of understanding what knowledge of these quantities tells us about the
motion.
Definition 27 (Speed and the unit tangent vector). The magnitude of the velocity vector is called
the speed. We denote it by v(t). That is,
v(t) = |v(t)| .
Provided that v(t) 6= 0, we can define a unit vector valued function T(t) by
T(t) =
1
v(t)
v(t)
so that
v(t) = v(t)T(t) .
The vector T(t) is called the unit tangent vector at time t. It specifies the instantaneous direction of
motion.
Example 27 (Speed and the unit tangent vector). Let x(t) = (t, 23/2 t3/2 /3, t2 /2) for t > 0 as in
Example 25. There we found that v(t) = (1, 21/2 t1/2 , t), and so the speed v(t) is given by
v(t) =
p
1 + 2t + t2 = 1 + t .
which is strictly positive for all t > 0, and then we have

T(t) =
1
(1, 21/2 t1/2 , t) .
1+t
Theorem 19. Let x(t) be a twice differentiable curve, and suppose that the speed v(t) is nonzero on
some open interval (b, c) so that T(t) is defined for all t in this interval. Then, the rate of change of
the speed, v 0 (t), is given by
v 0 (t) = a(t) T(t)
for all b < t < c.
Proof: Since v 2 (t) = v(t) v(t), we can apply Theorem 17 to conclude
2v(t)v 0 (t) = 2v(t) v0 (t) = 2v(t) a(t) = 2v(t)T(t) a(t) ,
and since v(t) 6= 0, we may divide through on both sides by 2v(t).
Now consider the decomposition of the acceleration a(t) into its components ak (t) and a (t),
parallel and orthogonal to T(t). By definition, ak (t) = (a(t) T(t))T(t). Thus, by Theorem 19,
ak (t) = v 0 (t)T(t). By (2.17),
a(t) = (v(t)T(t))0 = v 0 (t)T(t) + v(t)T0 (t) ,
and since ak (t) = v 0 (t)T(t), it follows that a (t) = v(t)T0 (t). We summarize this important consequence of Theorem 19:
ak (t) = v 0 (t)T(t)
and
a (t) = v(t)T0 (t) .
(2.20)
70

Thus the tangential component of the acceleration has to do with the rate of change of the speed,
while the normal component has to do with the rate of change of the direction of the velocity vector,
T(t).
In particular, when the speed is constant, so that v 0 (t) = 0, ak (t) = 0, and then only the normal
component of the acceleration can be non-zero. The following example is fundamentally important
for understanding the meaning of the normal component of the acceleration.
Example 28 (Constant speed circular motion). Let x(t) be the R2 valued function given by
x(t) = %( cos(t), sin(t))
q
for some fixed % > 0. Since kx(t)k = %2 (cos2 (t) + sin2 (t) = %, the motion described by this
curve runs over the circle of radius % in the plane. Differentiating, we get
v(t) = %( sin(t), cos(t))
a(t) = 2 %( cos(t), sin(t)) .
and
Thus,
v(t) = kv(t)k = %
T(t) = ( sin(t), cos(t)) .
and
Since the speed is constant, ak (t) = 0 for all t. Therefore,

a (t) = a(t) = 2 %( cos(t), sin(t)) ,
and ka (t)k = 2 % , and so the radius % of the circle is given by
%=
v 2 (t)
ka (t)k
for all t. Since for circular motion at constant speed, a(t) = a (t) for all t, this can be rewritten as
kak =
v2
.
%
Summarizing, for circular motion at constant speed, the magnitude of the acceleration is directly
proportion to the square of the speed, and inversely proportional to the radius of the circle. The
smaller the radius of the circle, the more curved the circle is.
The previous example motivates the following definition.
Definition 28 (Curvature and the unit normal vector). Let x(t) be a twice differentiable curve in
Rn , and suppose that the speed v(t) is nonzero on some open interval (b, c) so that T(t) is defined for
all t in this interval.
The curvature (t) at time t is defined by
(t) =
ka k
,
v 2 (t)
(2.21)
and the radius of curvature %(t) at time t is defined by

%(t) =
1
.
(t)
Furthermore, if ka k =
6 0, we define the unit normal vector N(t) by
N(t) =
1
a ,
ka k
(2.22)
71
Comparing (2.20) and (2.22), we see that N(t) points in the same direction as T0 (t). Thus, it
points in the direction in which the curve is turning.
Since T(t) is the direction of ak (t), and N(t) is the direction of a (t), T(t) and T0 (t) are
orthogonal. Here is another way to see this: Since T(t) is a unit vector,
T(t) T(t) = 1 .
Therefore, by Theorem 17,
d
1 = 2T(t) T0 (t) ,
dt
which shows the orthogonality of T(t) and T0 (t).
0=
Example 29 (Normal and tangential acceleration). Let x(t) = (t, (2t)3/2 /3, t2 /2) for t > 0. We
have computed in Example 27 that
v(t) = 1 + t
and
T(t) =
1
(1, (2t)1/2 , t) .
1+t
Therefore, v 0 (t) = 1, and so ak (t) = T(t). Thus,

ak (t) =
1
(1, (2t)1/2 , t) .
1+t
This is the tangential component of the acceleration. The normal component is a(t) minus this.
Since we have computed in Example 26 that
a(t) = x00 (t) = (0, (2t)1/2 , 1) ,
the normal component is
(0, (2t)1/2 , 1)
1
1
(1, (2t)1/2 , t) =
(1, (1 t)(2t)1/2 , 1) .
1+t
1+t
From here we compute
1
ka (t)k = .
2t
Hence
N(t) =
and
2t
( 1, (1 t)(2t)1/2 , 1)
1+t
(t) =
2t
(1 + t)2
and
%(t) =
(1 + t)2
.
2t
Notice that by combining (2.21) and (2.22), we obtain a simple formula for a (t), namely
a (t) = v 2 (t)(t)N(t)
(2.23)
Comparing (2.20) and (2.23), we deduce two useful formulas:

Theorem 20. Let x(t) be a twice differentiable curve in Rn . Then
a(t) = v 0 (t)T(t) + v 2 (t)(t)N(t) ,
(2.24)
T0 (t) = v(t)(t)N(t) .
(2.25)
and
72
Proof: Compare (2.20) and (2.23).

As a consequence of Theorem 20, we can deduce a simple and effective formula for computing
curvature in R3 :
Theorem 21 (Computing curvature in R3 ). Let x(t) be a twice differentiable curve in R3 such that
v(t0 ) > 0. Let v and a denote the velocity and acceleration at time t0 . Let v and denote the speed
and curvature at time t0 . Then
=
kv ak
.
v3
(2.26)
Proof: We compute:
va
(vT) (v 0 T + v 2 N)
= vv 0 T T + v 3 T N
= v 3 T N
T and N are orthogonal unit vectors, T N is a unit vector, and hence |v a| = v 3 .
The aspect of the formula (2.26) that is three dimensional is that it involves the cross product.
Nonetheless, it can also be applied in two dimensions: We can consider any planar curve in R2 as
a curve in R3 for which x3 (t) = 0 for all t. Thus everything we have deduced about curves in R3
applies to curves in R2 as a special case.
2.1.4
Torsion and the FrenetSeret formulae for a curve in R3
Curves in R3 are especially important since we live in a three dimensional world. In this case, we
can compute a unit normal to the plane through x(t0 ) that contains both T(t0 ) and N(t0 ) by taking
the cross product of T(t0 ) and N(t0 ).
Definition 29 (Binormal vector and osculating plane). Let x(t) be a twice differentiable curve in
R3 . Then at each t0 for which v(t0 ) 6= 0 and (t0 ) 6= 0, so that T(t0 ) and N(t0 ) are well defined, the
binormal vector B(t0 ) is defined by
B(t0 ) = T(t0 ) N(t0 ) ,
(2.27)
and the osculating plane at t0 is the plane specified by the equation

B(t0 ) (x x(t0 )) = 0 .
(2.28)
Since B(t0 ) is orthogonal to T(t0 ) and N(t0 ), (2.28) is the equation of the plane through x(t0 )
that contains both T(t0 ) and N(t0 ). Since v = vT and a = v 0 T + v 2 N, v a = v 3 B, which yields
the useful formulas
B(t0 ) =
1
1
v(t0 ) a(t0 ) =
v(t0 ) a(t0 ) .
v 3 (t0 )(t0 )
kv(t0 ) a(t0 )k
(2.29)
It follows that the direction of B is the same as that of v a. Therefore, the osculating plane at time
t0 is the plane through x(t0 ) that contains v(t0 ) and a(t0 ). For this reason, the osculating plane is
73
sometimes called the instantaneous plane of motion, and another equation for the osculating plane
at t = t0 is
(v(t0 ) a(t0 )) (x x(t0 )) = 0 .
In particular, it is not necessary to go through all the work of computing T, N and then B if all you
wanted to find was an equation for the osculating plane. You can find the equation directly from a
computation of v, a and v a.
We emphasize that we are assuming throughout these paragraphs, as in Definition 29, that
v(t0 ) 6= 0 and (t0 ) 6= 0, so that T(t0 ) and N(t0 ) are well defined. Otherwise, it does not make sense
to refer to the plane through x(t0 ) containing v(t0 ) and a(t0 ).
Example 30 (An osculating plane). Let x(t) = (t, 23/2 t3/2 /3, t2 /2) for t > 0. In Example 26, we
have computed that
x(1) = (1, 23/2 /3, 1/2)
v(1) = (1, 21/2 , 1)
and
a(1) = (0, 21/2 , 1) .
We now compute
v(1) a(1) = (21/2 , 1, 21/2 ) .
The equation for the osculating plane then is
(21/2 , 1, 21/2 ) (x 1, y 23/2 /3, z 1/2) = 0
which reduces to
x 21/2 y + z = 6 .
Definition 30 (Planar curve in R3 ). A curve x(t) in R3 , a < t < b, is planar in case there is some
plane in R3 that contains x(t) for all a < t < b. In other words, x(t) is planar in case there exists a
non-zero vector n and a constant d such that n x(t) = d for all a < t < b.
Planar curves are easy to recogonize when the plane is one of the coordinate planes: For example
x(t) := (t, t2 , 0) is clearly a planar curve a parabola in the x, y plane - for which we may take
n = e3 and d = 0. But planar curves are not always so easy to recognize. Consider the curve
x(t) := ( 1 + 2t t3 , t + 3t2 2t3 , 1 + 2t 6t2 + 2t3 ) .
(2.30)
As we shall see, this is in fact a planar curve. How can we recognize that, and what is the plane that
contains the curve?
Theorem 22 (The binormal vector and planar curves). Let x(t) be a twice differentiable curve for
a < t < b such that v(t) and (t) are non-zero for all a < t < b. Then x(t) is planar if and only if
B(t) is a constant vector on (a, b). In this case, there is exactly one plane containing the curve, and
for any a < t0 < b, if we define n = B(t0 ) and d = x(t0 ) B(t0 ), then n x = d is an equation for the
unique plane containing the curve.
Proof: Suppose that B(t) is constant on (a, b). Then for all a < b < t, and any t0 (a, b)
(x(t) B(t0 ))0 = x0 (t) B(t0 ) = v(t) B(t) =
1
v(t) (v(t) a(t)) = 0
kv(t) a(t)k
74
by the triple product identity. Hence for all t (a, b)

x(t) B(t0 ) = x(t0 ) B(t0 ) .
This shows that with n := B(t0 ) and d := B(t0 ) x(t0 ), the plane specified by n x = d contains
x(t) for all t (a, b). There can be no other plane containing the curve since the intersection of two
distinct planes in R3 is either empty or is a line. Since by hypothesis x(t) has non-zero curvature, it
is not contained in any line. Hence the plane is unique.
On the other hand, suppose that x(t) is planar, and therefore satisfies n x(t) = d for some
non-zero n and some d. Differentiating twice we obtain
0 = (n x(t))0 = n v(t)
0 = (n v(t))0 = n a(t) .
and then
Hence for all t (a, b), n is orthogonal to both v(t) and a(t), Since by hypothesis the curvature is
non-zero, v(t) and a(t) are not multiples of one another, and so
B(t) =
1
1
v(t) a(t) =
n.
kv(t) a(t)k
knk
Since B(t) is continuous, the sign cannot change anywhere in (a, b), and so B(t) is constant whenever
the curve is planar.
Example 31 (Identifying a planar curve). Let x(t) be given by (2.30). Let us compute B(t) for this
curve, and see whether it is constant or not. Before beginning the computation, it will pay to regroup
the entries in x(t). Note that
x(t) = w0 + tw1 + t2 w2 + t3 w3
where
w0 := ( 1, 0, 1) ,
w1 := (2, 1, 2) ,
w2 := (0, 3, 6) ,
and
w3 := ( 1, 1, 2) .
Then we have
v(t) = w1 + 2tw2 + 3t2 w3
and
a(t) = 2w2 + 6tw3 .
Therefore
v(t) a(t)
(w1 + 2tw2 + 3t2 w3 ) (2w2 + 6tw3 )
2(w1 + 3t2 w3 ) w2 + 6t(w1 + 2tw2 ) w3
2w1 w2 + 6tw1 w3 + 6t2 w2 w3 .
We then compute
w1 w2 = 6( 2, 2, 1) ,
w1 w3 = 3( 2, 2, 1) ,
and
w2 w3 = 3( 2, 2, 1) .
Altogether then
v(t) a(t) = (12 18t + 18t2 )( 2, 2, 1)
and hence
B(t) =
1
( 2, 2, 1) .
3
Since x(0) = ( = 1, 0, 1), B(0) x(0) = 1. Thus, the plane containing the curve satisfies the equation
2x + 2y + z = 1 .
75
As we have seen in our examples so far, the rate of change of the basis T(t) and B(t) tell us
important information about the shape of a curve: The curvature (t) is related to T0 (t) through
T0 (t) = v(t)(t)N(t), and the curve is planar if and only if B0 (t) = 0 for all t. But this only scratches
the surface. There is much more to be learned by considering the rates of change of the vectors in
the right handed orthonormal basis
{T(t), N(t), B(t)}
that is carried along by any twice differentiable curve in R3 with non-zero speed and curvature.
First, let us consider B0 (t).

Lemma 6. Let x(t) be a twice differentiable curve in R3 with non-zero speed and curvature. Then
for each t, B0 (t) is a multiple of N(t).
Proof of Lemma 6: Since {T(t), N(t), B(t)} is an orthonormal basis, by Theorem 5,
B0 = (B0 T)T + (B0 N)N + (B0 B)B ,
where we have dropped the explicit t dependence to make the formulas more readable. Thus, it
suffices to prove that B0 T = 0 and B0 B = 0.
But B T0 = B (vN) = 0, and so B0 T = 0. Also, for each t, B B = 1, and so
0 = (B B)0 = 2B0 B ,
which proves that B0 B = 0.
We now define torsion, (t), which quantifies the rate of change of the binormal vector B(t), and
therefore quantifies the extent to which the curve is twisting out of its osculating plane:
Definition 31 (Torsion). Let x(t) be a twice differentiable curve in R3 with non-zero speed and
curvature for all t (a, b). Then the torsion at t (a, b) is the quantity (t) defined by
B0 (t) = v(t) (t)N(t) .
(2.31)
T0 (t) = v(t)(t)N(t) .
(2.32)
We have already seen that
Notice the similarity between (2.31) and (2.32). The resaon for including the minus sign in (2.31)
will become evident later.
76
Theorem 23 (Computing torsion in R3 ). Let x(t) be a thrice differentiable curve in R3 such that
v(t0 ) > 0 and (t0 ) > 0. Let v and a denote the velocity and acceleration at time t0 . Let v and
denote the speed and curvature at time t0 . Then
=
x000 x00 x0
a0 v a
=
.
v 6 2
v 6 2
(2.33)
Proof: We start from the formula v a = v 3 B that was derived in (2.27). Differentiating both
sides we find
(v a)0
(v 3 )0 B + (v 3 )B0
(v 3 )0 B v 4 N .
Taking the dot product of both sides with a = v 0 T + v 2 N yields

a (v a)0 = 2 v 6 .
Solving for , we obtain the first formula in (2.33). Next, since a (v a) = 0 for all t, 0 =
(a (v a))0 = a0 (v a) + a (v a)0 and so a0 (v a) = a (v a)0 .
Finally, let us derive a formula for N0 (t):
Lemma 7. Let x(t) be a thrice differentiable curve in R3 with non-zero speed and curvature for all
t (a, b). Then for all t (a, b),
N0 (t) = v(t)(t)T(t) + v(t) (t)B(t) .
(2.34)
Proof: Since {T(t), N(t), B(t)} is an orthonormal basis, Theorem 5 gives us

N0 = (N0 T)T + (N0 N)N + (N0 B)B ,
(2.35)
where we have dropped the explicit t dependence to make the formulas more readable.
We now claim that N0 N = 0 and that
N0 T = N T0
and N0 B = N B0 .
(2.36)
Assuming this for the moment, (2.35) simplifies to

N0 = (N T0 )T (N B0 )B .
(2.37)
By (2.32) and (2.31), N T0 = v and N B0 = v . Substituting these into (2.37) yields (2.34).
We now verify N0 N = 0 and (2.36). Since for each t, N N = 1,
0 = (N N)0 = 2N0 N .
so that N0 N = 0.
Likewise, for each t, N T = 0 = N B, and hence
0 = (N T)0 = N0 T + N T0 ,
and
0 = (N B)0 = N0 T + N B0 .
This gives us the two identities in (2.36).
Summarizing the results, we have proved the following:
77
Theorem 24 (FrenetSeret formulae). Let x(t) be a thrice differentiable curve in R3 with non-zero
speed and curvature at each t in some open interval so that T(t), N(t) and B(t) are all defined and
differentiable on this interval. Then for all t in this interval,
T0 (t)
0
v(t)(t)N(t)
N (t)
= v(t)(t)T(t) + v(t) (t)B(t)
B0 (t)
= v(t) (t)N(t) .
There is a more useful way to express these three formulae.

Definition 32 (Darboux vector). Let x(t) be a twice differentiable curve with non-zero speed and
curvature at each t in some open interval so that T(t), N(t) and B(t) are all defined on this interval.
The Darboux vector is defined on this interval by by
(t) = (t)T(t) + (t)B(t) .
The point of the definition is that since {T, N, B} is constructed to be a right-handed orthonormal
basis of R3 , Theorem 9 says that
TN=B
NB=T
BT=N ,
and
and thus,
T =
( T + B) T =
( T + B) N = T + B
B =
( T + B) B = N .
Comparing with Theorem 2.38, we see that

T0 (t)
= v(t)(t) T(t)
N0 (t)
= v(t)(t) N(t)
B0 (t)
= v(t)(t) B(t) .
(2.38)
As we shall see later in this chapter, this means that for small h > 0, the orthonormal basis
{T(t + h), B(t + h), B(t + h)} is, up to errors of size h2 , what one would get by applying a rotation
of angle v(t)k(t)k about the axis of rotation in the direction of (t). That is, the Darboux vector
describes the instantaneous rate and direction of rotation of the orthonormal basis {T(t), B(t), B(t)}.
Example 32 (Curvature and torsion for helices). Consider the curve x(t) given by
x(t) := (r cos t, r sin t, bt)
for some r > 0 and b 6= 0. This curve is a helix: There is circular motion in the x, y variables, and
linear motion in the z variable. A plot of the curve will look something like:
78
The plot was made using the values r = 3 and b = 1 for 0 t 9.

Let us compute the curvature, torsion, the orthonormal basis {T(t), N(t), B(t)} and the Darboux
vector (t). To begin, we compute
v(t) = ( r sin t, r cos t, b)
from which it follows that
v(t) =
r 2 + b2
and
T(t) =
r
( sin t, cos t, b/r) .
r2 + b2
We next compute
a(t) = ( r cos t r sin t, 0) .
Then since v(t) a(t) = 0, the parallel component of the acceleration is zero, and so a (t) = a(t).
Since N(t) is the normalization of a (t), and hence in this case of a(t), it follows that
ka (t)k = ka(t)k = r
and
N(t) = ( cos t, sin t, 0) .
The curvature is
(t) :=
ka (t)k
r
= 2
.
2
v (t)
r + b2
Let us pause to note that this is reasonable: If b = 0, the helix is simply a circle of radius r in the x, y
plane, and so as b approaches zero, we must have that the curvature approaches 1/r. On the other
hand, if b is very large, the motion is essentially vertical, and the curvature is very small. This is in
agreement with the formula we have found.
We next compute
v(t) a(t) = rb( sin t, cos t, r/b) .
Hence
B(t) =
1
b
v(t) a(t) =
( sin t, cos t, r/b) .
2
kv(t) a(t)k
r + b2
79
Since B(t) and N(t) are so simple in this case, the easiest way to compute the torsion (t) is directly
from the defining relation B0 (t) = v(t) (t)N(t). We compute
B0 (t) =
r2
b
b
( sin t, sin t, 0) =
N(t) ,
2
2
+b
r + b2
Thus
b
b
= v(t) (t)
so that
(t) = 2
.
r + b2
r2 + b2
Of course, you could also have computed the curvature and torsion using the formulas from The
orems 21 and 23 respectively. However, those formulae do not always provide the simplest approach.
Notice that both the curvature and the torsion turn out to be constant. Let us now compute the
Darboux vector (t):
(t) = (t)T(t) + (t)B(t) =
r2
rb
(b/r + r/b)(0, 0, 1) .
+ b2
Notice that this, too, is constant, despite the fact that neither T(t) nor B(t) are constant.
Here is a plot, once more for r = 3 and b = 1 for 0 t 9, but this time showing {T(t), N(t), B(t)}
for t = 7 and t = 9:
2.1.5
Curvature and torsion are independent of parameterization.
The same path can be parameterized many ways. For example, consider
x(t) = ( cos(t), sin(t))
and
y(u) = ( cos(u3 ), sin(u3 )) .
As t and u vary over R, both of these curves trace out the unit circle in R2 , but they trace it out in
different speeds, and one traces it out counterclockwise, and the other clockwise.
Definition 33 (Reparameterization). Let x(t) be a curve in Rn defined on an open interval (a, b)
R, and let y(u) be another curve in Rn defined on an open interval (c, d) R. Either a or c may be
, and either b or d may be +. Then y(u) is a reparameterization of x(t) in case there is a a
continuous, strictly monotone increasing or decreasing function t(u) from (c, d) onto (a, b) such that
y(u(t)) = x(t)
for all
t (a, b) .
80
Example 33. Define t(u) = u3 and u(t) = t1/3 . Then with x(t) = ( cos(t), sin(t)) and y(u) =
( cos(u3 ), sin(u3 )), we have both
y(u) = x(t(u))
uR
for all
and
x(t) = y(u(t))
for all
uR.
Thus the x(t) and y(u) are reparameterizations of each other, and they both parameterize the unit
circle.
As in the example, whenever y(u) is a reparameterization of x(t), then x(t) is a reparameterization of y(u). Indeed, if t(u) is any continuous, strictly monotone increasing function t(u) from (c, d)
onto (a, b), then it is both one-to-one and onto, and so it has an inverse function u(t) from (c, d) to
(a, b) which is also continuous and strictly monotone increasing.
It turns out that while any curve can be parameterized in inifinitely many ways, the curvature at a
point on the path is a purely geometric property of the path traced out by the curve it is independent
of the parameterization. Not only that, so is the unit normal vector, and, up to a sign, so is the unit
tangent vector.
To see this suppose that x(t) and y(u) are two parameterizations of the same path in Rn . Suppose
that
x(t0 ) = y(u0 )
so that when t = t0 and u = u0 , both curves pass through the same point. Let us suppose also that
the two parameterizations are related in a smooth way, so that t(u) is twice continuously differentiable
in u.
Then, by the chain rule,
d
d
y(u) =
x(t(u)) =
y (u) =
du
du
0
dt
du
x0 (t(u)) .
Evaluating at u = u0 , and recalling that t0 = t(u0 ), we get the following relation between the speeds
at which the two curve pass through the point in question:

dt
0
ky (u0 )k = kx0 (t0 )k .
du
Therefore,
1
y0 (u0 )
ky0 (u0 )k
!
1
dt dt
1
0

du du kx(t0 )k x (t0 )
1
x0 (t0 ) .
kx(t0 )k
The plus sign is correct is t is an increasing function of u, in which case the two parameterizations
trace the path out in the same direction, and otherwise the minus sign is correct.
This shows that up to a sign, the unit tangent vector T at the point in question comes out the
same for the two parameterizations.
81
Next, let us differentiate once more. We find

d 0
d
dt
00
0
y (u) =
y (u) =
x (t(u))
du
du
du
2
2
dt
d t
0
x (t(u)) +
x00 (t(u)) .
=
2
du
du
Evaluating at u = u0 , and recalling that t0 = t(u0 ), we find the following formula relating the
acceleration along the two curves as they pass though the point in question:
00
y (u0 ) =
d2 t
du2
x (t0 ) +
dt
du
2
x00 (t0 ) .
Notice that the first term on the right is a multiple of T, and hence when we decompose y00 (u0 )
into its tangential and orthogonal components, this piece contributes only to the tangential component. Hence
00
y
(u0 )

=
dt
du
2
x00 (t0 ) .
00
Because of the square, y
(u0 ) is a positive multiple of x00 (t0 ), and so these two vectors point in the
exact same direction. That is,

N=
1
y00 (u0 )
00
ky (u0 )k
1
x00 (t0 )
00
kx (t0 )k
showing that the normal vector N is independent of the parameterization.

Next, we consider the curvature. Since
1
ky00 (u0 ))k
0
ky (u0 )k2
2
2
dt
1
dt
kx00 (t0 )k
0
2
du
|x (t0 )|
du
1
kx00 (t0 )k ,
kx0 (t0 )k2

=
=
we get the exact same value for the curvature at the same point, using either parameterization. This
shows that although in practice we use a particular parameterization to compute the curvature and
the unit normal N, the results do not depend on the choice of the parameterization, and are in fact
an intrinsically geometric property of the path that the curve traces out.
So far what we have said about reparameterization is valid in Rn for all n 2. In R3 , there is
more to say. In R3 , we also have the binormal vector B = T N and the torsion .
Since B(t) = T(t)N(t), it follows that B(t) is well defined, independent of the parameterization,
up to a sign. Then, consideration of the formula
B0 (t) = v(t) (t)N(t)
under two parameterization shows that like the curvature, the torsion is independent of the parameterization, up to a sign. The calculations that show this are very similar to the calculuations we
have just made for , T and N, and are left to the reader. The conclusion is that the torsion also is
determined by the geometry of the path itself, and not how fast or slow we move along it.
82
2.1.6
Speed and arc length
The speed v(t) represents the rate of change of the distance traveled with time. Given some reference
time t0 , define
Z
s(t) =
v(u)du .
(2.39)
t0
Then by the Fundamental Theorem of Calculus,

d
s(t) = v(t)
dt
and clearly s(t0 ) = 0. Hence the rate of change of s(t) is v(t), which is the rate of change of the
distance traveled with time, as one has moved along the path traced out by x(t).
Definition 34 (Arc length). The function s(t) defined by (2.39) is called the arc length along the
path traced out by x(t) since time t0 .
Example 34 (Computation of arc length). Let x(t) be given by x(t) = (t, 23/2 t3/2 /3, t2 /2) for t > 0
as in Example 26. Then, as we have seen, for all t > 0, v(t) = 1 + t. Therefore,
Z t
t2
.
s(t) =
(1 + u)du = t +
2
0
If you took a piece of string, and cut it so it can be run along the path from the starting point to the
position at time t, the length of the string would be t + t2 /2 units of distance.
By definition, v(t) 0, and so s(t) has a non-negative derivative. This means that it is an
increasing function. As long as v(t) > 0; i.e., as long as the particle never comes to even an instantaneous rest, s(t) is strictly monotone increasing. Let us suppose that for some t1 > t0 , v(t) > 0 for
all t0 < t < t1 . Then s(t) is strictly monotone increasing on the interval [t0 , t1 ].
Then for each s [s(t0 ), s(t1 )], there is exactly one value of t [t0 , t1 ] so that
s(t) = s .
(2.40)
This value of t, considered as a function of s, is the inverse function to the arc length function:
t(s) = t .
(2.41)
It answers a very simple question, namely: How much time will have gone by when the distance
travelled is s units of length?
If you can compute an explicit expression for s(t), such as the result s(t) = t + t2 /2 that we
found in Example 9, what you then need to do to answer the question is to find the inverse function
t(s); i.e., to solve (2.40) to find t in terms of s:
Example 35 (Time as a function of arc length). Let x(t) be given by
x(t) = (t, 23/2 t3/2 /3, t2 /2) as in Example 34. Then, as we have seen, for all t > 0, s(t) = t + (t2 /2).
To find t as a function of s, write this as
s=t+
t2
2
83
and solve for t in terms of s. In this case,

t+
so t =
t2
1
= ((t + 1)2 1)
2
2
2s + 1 1. That is,
t(s) =
2s + 1 1 .
This function tells you how long it took to travel a given distance s when moving along the curve.
We can then get a new parameterization of our curve by defining x(s) by
x(s) = x(t(s)) .
This is called the arc length parameterization. We have changed our habits of notation somewhat:
Now we use the sme letter x for both parameterizations to emphasize that they are two parameterizations of the same curve.
Example 36 (Arc length parameterization). Let x(t) = (t, 23/2 t3/2 /3, t2 /2) as in Example 35. Then,
as we have seen, for all t > 0, t(s) = 2s + 1 1 Therefore,
x(s) = x(t(s)) = ( 2s + 1 1, 23/2 ( 2s + 1 1)3/2 /3, ( 2s + 1 1)2 /2) .

The arc length parameterization generally is complicated to work out explicitly. Even when you
can work it out, it often looks a lot more complicated than whatever t parameterization you started
with. So what is it good for?
The point about the arc length parameterization is that it is purely geometric, so that it helps
us to understand the geometry of the path that a parameterized curve traces out. If we compute the
rate of change of the unit tangent vector T as a function of s, we are computing the rate of turning
per unit distance along the curve. This is an intrinsic property of the curve itself. If we compute rate
of change of the unit tangent vector T as a function of t, we are computing something that depends
on how fast we are moving on the curve, and not just on the curve itself. Indeed, if we use the arc
length parameterization, v(s) = 1 for all s, and so the factors involving speed drop out of all of our
formulas. For example,
d
x(s) = T(s)
ds
and
d
T(s) = (s)N(s) .
ds
Often, this last formula is taken as the definition of the normal vector N and curvature . The
advantage of this definition is that it is manifestly geometric, so that the normal vector N and
curvature do not depend on the parameterization of the curve. The disadvantage is that it is
generally very difficult to explicitly work out the arc length parameterization. In order to more
quickly arrive at computational examples, we have chosen the form of the definition that is convenient
for computation.
84
2.2
2.2.1
The prediction of motion

Newtons Second Law
Newtons Second Law states that if x(t) denotes the position at time t of a particle of mass m, and
at each time t a force F(t) is acting on the particle, the acceleration of the particle, a(t), satisfies
a(t) =
1
F(t) .
m
(2.42)
For example, a particle of mass m in R3 that is subject to a constant gravitational field (pointing
downwards by convention) is acted upon by the force
F = mg(0, 0, 1) ,
where g is the gravitational constant, and is about 9.8 meters/second2 on the Earth.
Since a(t) is the derivative of the velocity v(t) = (v1 (t), v2 (t), v3 (t)), this means that
v0 (t) = g(0, 0, 1) .
(2.43)
This is a differential equation for v(t), it tells us what the derivative of v(t) is. (The term derivative
equation might be more descriptive, but it is not what is used.) The kind of motion described by(2.43)
is called ballistic motion.
The differential equation is very simple; the right hand side is constant. If we are also given the
initial value v(0) = v0 , we can solve for v(t) using the Fundamental Theorem of Calculus, entry by
entry.
Z
Given a vector a continuous vector valued function w(t) in R , define
w(t)dt to be the vector

a
in R whose jth component is

Z
wj (t)dt .
0
That is, we simply integrate w(t) entry by entry. Then since we also differentiate entry by entry, the
fundamental Theorem of Calculs, applied entry by entry, gives us that
Whenever z(t) is a continuously differentiable function in Rn for a t b,
Z
z(b) = z(a) +
z0 (t)dt .
(2.44)
Applying this to ballistic motion, if the initial velocity is v0 , then

Z
g(0, 0, 1)dr = v0 tg(0, 0, 1) .
v(t) = v0 +
0
Since this gives an explicit expression for x0 (t) = v(t), we can apply (2.44) once more to obtain
Z
0
where x0 is the initial position.
[v0 rg(0, 0, 1)]dr = x0 + tv0
x(t) = x0 +
gt2
(0, 0, 1) ,
2
2.2. THE PREDICTION OF MOTION
85
That is, we have solved Newtons Equation to deduce that the particles trajectory is given by
1
x(t) = x0 + tv0 gt2 e3 .
2
(2.45)
The term ballistic motion is used to describe the motion along such a trajectory where some
launching process imparts the initial velocity, and after that the only force acting is gravity. Once
you know the initial position and velocity, you know the whole trajectory, and can determine (or
predict) any aspect of the motion. For example, suppose that the initial position is at the origin;
i.e., x(0) = 0, and that v(0) = (a, 0, b) with a, b > 0. This corresponds to launching the particle
with this initial velocity, upwards and outwards along the e1 axis. What is the maximum height of
the particle along the trajectory? What is the distance from the origin when it hits the ground?
Example 37 (Making predictions for ballistic motion). Let us answer the questions raised just above
concerning ballistic motion. To do this, plug our initial position 0 and initial velocity (a, 0, b) into
(2.45). We find that the height of the particle at time t, x3 (t), is given by
1
x3 (t) = tb gt2 .
2
Completing the square,
x3 (t) =
g
2
b
t
g
2
+
1 b2
.
2 g
(2.46)
The maximum height is achieved when the first term is zero, i.e., at t = b/g, at which time the height
is b2 /(2g).
The particle hits the ground when x3 (t) = 0 for the second time. From (2.46), we see that
x3 (t) = 0 if and only if

2
b
b2
t = 2 .
g
g
The two solutions are t = t0 = 0 and t = t1 = 2b/g. Since t0 = 0 is the launch time, t1 = 2b/g is the
time when the particle hits the ground. Evaluating x(t1 ) at this time t, we find

2ab
x(t1 ) =
, 0, 0 .
g
Thus, the distance from the origin to where the particle hit the ground is 2ab/g.
Let us go a little further with this, suppose that however you are launching the particle, the
maximum launch speed you can achieve is 10 meters/second. Then, to have the particle hit the
ground as far away along the e1 axis as possible, you should use an initial velocity vector (a, 0, b) with
a2 + b2 = 100, but which one?
Since a2 + b2 2ab = (a b)2 , we have 2ab a2 + b2 , with equality if and only if a = b. Thus to
maximize 2ab under the constraint a2 + b2 = 100, you should use a = b = 50 meters/second. This
corresponds to the particle at an initial angle of /4 with the horizontal.
2.2.2
Ballistic motion with friction
The equation (2.43) only provides an accurate prediction for the motion of a projection only when
the effects of friction are negligible. Frictional forces, for moderate velocities, can taken into account
86
with a useful degree of accuracy by postulating a frictional force of the form

Ffric := v ,
(2.47)
where > 0 is a proportionality constant, and v is the velocity. The minus sign says that the frictional
force opposes the motion of direction, whatever it is, and that the magnitude of the frictional force
is proportional to the magnitude of the velocity; i.e., the speed. Adding the frictive force to the
gravitational force, and applying Newtons Second Law, we obtain the differential equation
v0 (t) = g(0, 0, 1)
v(t) .
m
(2.48)
This is more complicated than (2.43) since in that equation, the right hand side was an explicit
constant vector, and here, the right hand side depends on the unknown vector v(t). However, there
is a standard device for reducing (2.48) to the constant right hand side case. It is a very simple
device, but very important: We shall use it again and again. Here it is: Regroup the terms in (2.48)
and multiply through by et(/m) to obtain
i
h
et(/m) v0 (t) + v(t) = et(/m) g(0, 0, 1) .

m
(2.49)
Defining the new dependent variable

z(t) := et(/m) v(t) ,
we see that (2.50) is equivalent to
z0 (t) = et(/m) g(0, 0, 1) .
(2.50)
Now, the right hand side is not constant, but it is an explicit function of t that is easily integrated,
so that, once again, we may apply the Fundamental Theorem of Calculus:
Z t
z(t) z(0) =
et(/m) g(0, 0, 1)dr
0
mg
=
(0, 0, et(/m) 1) .
Since v(0) = z(0), and more generally, v(t) = et(/m) z(t), we have
v(t) = et(/m) v(0)
mg
(0, 0, 1 et(/m) ) .
Notice that
mg
(0, 0, 1) .
This is the terminal velocity for ballistic motion with this frictive coefficient .
Rt
Next, since x(t) = x(0) + 0 v(r)dr, we have
i
i
m h t/m
mg
m h t(/m)
x(t) = x(0)
e
1 v(0)
0, 0, t +
e
1
.
lim v(t) =
Now let consider the case in which x(0) = 0, and that v(0) = (a, 0, b) with a, b > 0. Then with
x(t) = (x(t), y(t), z(t)), we have
x(t)
z(t)
i
m h t/m
e
1 a
i
i
m h t/m
mg
m h t(/m)
e
1 b
t+
e
1
,
87
and of course y(t) = 0 for all t. Using the Taylor approximation es 1 + s + s2 /2 which is valid for
small values of s, and the formulas derived above, we see that
lim x(t) = at
lim z(t) = bt gt2 /2 ,
and
as we had found in the frictionless case.

Let us suppose once again that we are launching a particle of mass m on ballistic motion with
friction coefficient , and the maximum launch speed we can obtain is again 10 meters per second.
If we want to particle to travel as far as possible before hitting the ground, and what angle should
we launch it?
The particle hits the ground when z(t) = 0. However, this is now a transcendental equation, and
we cannot simply solve for t to pug into x(t). Instead, we write (a, b) = 10( cos , sin ), and define
i
m h t/m
e
1 cos
f (t, ) := 10
i
i
m h t/m
mg
m h t(/m)
g(t, ) := 10
e
1 sin
t+
e
1
.
Since by the formulas derived above, f (t, ) and g(t, ) are the horizontal distance traveled and the
height, respectively, at time t when the launch angle is , the problem is to maximize f (t, ) subject
to the constraint that g(t, ) = 0. That is, we seek to maximize f (t, ) not over the set of all values of
(t, ), but only over those for which g(t, ) = 0. This is a classic problem in the theory of constrained
optimization, and we will solve it later when we discuss the method of Lagrange multipliers. Our
present emphasis, though, is on the derivation of the trajectory from Newtons Second Law.
2.2.3
Motion in a constant magnetic field and the Rotation Equation
Ballistic motion is particularly simple, but is fundamentally important. Another fundamentally

important example concerns the motion of a charged particle in a constant magnetic field H. If the
particle has charge q and mass m, the Lorentz force law says that force F(t) due to the magnetic
field that acts on the particle at time t is F(t) = qv(t) H, and then Newtons Second Law yields
v0 (t) =
q
v(t) H .
m
(2.51)
Notice that the magnetic force is zero if the particle is not moving, or is moving parallel to H. We
shall solve (2.51) to find the particles trajectory.
First, let us simplify our notation. Define a vector b R3 by b =
q
H so that (2.51) becomes
m
v0 (t) = b v(t) ,
(2.52)
and we assume b 6= 0 to avoid trivialities.

In fact, this equation comes up all the time: We have already met it in different notation: Each
of the three Frenet-Seret equations (2.38) written in terms of the Darboux vector is of this form.
We shall now find all solutions of (2.52) that satisfy the initial condition
v(0) = v0
(2.53)
88
where v0 is a given initial value of the velocity vector. Together, (2.52) and (2.53) specify an initial
value problem. Once we have the solution in hand, we will see exactly why this equation comes up
all the time.
The equation (2.52) is not so simple as the equation for ballistic motion with friction since due
to the cross product, for each j, vj0 (t) depends on each of vk (t) for k 6= j: The equation mixes up
the entries of v(t). Thus, we cannot so easily separate variables as we did in the last subsection.
However, it is not a complicated matter to solve (2.52) and obtain an explicit formula for v(t).
First of all, we show that if v(t) solves (2.52) then kv(t)k = kv(0)k for all t; i.e., that that v(t)
is a constant of the motion. To see this, we differentiate
d
kv(t)k2 = 2v(t) v0 (t) = 2v(t) (b v(t)) = 0
dt
by the triple product identity.
This has an important consequence, namely that there is at most one solution to our initial
value problem. Indeed if v(t) and w(t) are any two solutions of the initial value problem, define
z(t) = v(t) w(t), and note that z0 (t) = b z(t). Indeed,
z0 (t) = v0 (t) w0 (t) = b (v(t) w(t) = b z(t) .
By what we have seen just above, this means that kz(t)k is constant. But since v(t) and w(t) have the
same initial value, namely v0 , z(0) = 0. Therefore, kz(t)k = kz(0)k = 0 for all t, and consequently,
v(t) = w(t) for all t: The two solutions are in fact the same. Hence, if we can find one solution of
our initial value problem, it is the only solution there is, and the problem is completely solved.
There is a second constant of the motion, namely b v(t). If v(t) solves v0 (t) = b v(t), then
(b v(t))0 = b (b v(t)) = (b b v(t) = 0 .
Then, since the component of v(t) parallel to b, vk (t) is given by
vk (t) =
1
(b v(t))b ,
kbk2
it follows that vk (t) = vk (t0 ) for all t. That is, vk is a constant of the motion. This naturally means
that kv(t)k k is constant, and since
v 2 (t) = kv(t)k2 = kvk v(t)k2 + kv (t)k2 ,
kv (t)k is constant as well.
For any solution v(t) of (2.52), the vector vk (t) and the quantity kv (t)k are constant. As t varies,
only the direction of v (t) changes.
To take advantage of our constants of the motion, we first introduce an orthonormal basis
{u1 , u2 , u3 } that is adapted to the problem at hand. We build this out of two orthogonal vectors
that come along with the problem, namely b itself, and (v0 ) , the component of v0 that is orthogonal
to b. Notice that if (v0 ) = 0, then b v0 = 0, and the initial value problem has the simple solution
v(t) = v0 for all t. By what we have said above, this is the only solution in this case.
89
Therefore, without loss of generality, we assume (v0 ) 6= 0, and define

u3 =
1
b
kbk
and
u1 =
1
(v0 ) .
k(v0 ) k
Once u3 and u1 are defined, we have only one choice for u2 that makes {u1 , u2 , u3 } a right-handed
orthonormal basis, namely u2 = u3 u1 .
Now let ((y1 (t), y2 (t), y3 (t)) be the coordinate vector for v(t) with respect to the basis {u1 , u2 , u3 }.
Then,
v(t) = y1 (t)u1 + y2 (t)u2 + y3 (t)u3 .
The merit of this coordinate system based on {u1 , u2 , u3 } is that
v (t) = y1 (t)u1 + y2 (t)u2
and
vk (t) = y3 (t)u3 .
But since vk (t) = vk (0) = (v0 )k , we have

v(t) = y1 (t)u1 + y2 (t)u2 + (v0 )k ,
(2.54)
and only y1 (t) and y2 (t) remain to be determined. For this we return to the equation (2.52): Differentiating both sides of (2.54) and taking the cross product of both sides with b, we deduce from
(2.52) that
y10 (t)u1 + y20 (t)u2
= y1 (t)b u1 + y2 (t)b u2
= kbk(y1 (t)u2 y2 (t)u1 )
where we have used b = kbku3 several times. Equating the coefficients of u1 on both sides, and
doing the same for u2 , we conclude
y10 (t) = kbky2 (t)
and
y20 (t) = kbky1 (t) .
(2.55)
Also from (2.54), we have that y1 (0)u1 + y2 (0)u2 = (v0 ) = k(v0 ) ku1 , we have
y1 (0) = k(v0 ) k
and
y2 (0) = 0 .
(2.56)
This gives us what we need to determine y1 (t) and y2 (t): We know that
y12 (t) + y22 (t) = kv (t)k2 = k(v0 ) k2
so that
For each t, (y1 (t), y2 (t)) is a point on the centered circle of radius k(v0 ) k2 .
Thus, for some function (t),
(y1 (t), y2 (t)) = k(v0 ) k( cos (t), sin (t)) .
Differentiating both sides, we find
y10 (t) = 0 (t)y2 (t)
and
y20 (t) = 0 (t)y1 (t) .
(2.57)
90
Comparing with (2.55) we see that 0 (t) = kbk for all t, and so (t) = kbkt+0 vfor some 0 [0, 2).
Using this in (2.57), and evaluating at t = 0, we find
(y1 (0), y2 (0)) = k(v0 ) k( cos(0 ), sin(0 )) .
Comparing with (2.56) w see that 0 = 0, so (t) = kbkt.
Now that we know y1 (t) and y2 (t), we know v(t). Altogether, the solution of (2.52) satisfying
(2.53) is
v(t) = k(v0 ) k(cos(kbkt)u1 + sin(kbkt)u2 ) + (v0 )k .
(2.58)
Now let us look at our explicit solution (2.58), and see what it is: The component vk (t) of v(t)
parallel to b stays constant, while the component (v0 ) (t) of v(t) orthogonal to b simply rotates in
the plane through 0 that is orthogonal to b, and does so at the constant angular velocity kbk, by
which we mean 0 (t) = kbk for all t. We conclude from our formula (2.58):
The equation v0 (t) = b v(t) describes rotation about an axis in the direction of b at constant
angular velocity kbk. If one observes the rotation in the plane orthogonal to b from a point along the
positive axis in the direction of b, the rotation is counter-clockwise.
Here is a plot showing an axis of rotation b, and initial vector v0 , and the curve swept out by
v(t) up the time it almost comes back to the starting point. As you see, when viewed from out along
the axis of rotation, the direction of rotation is counter-clockwise.
Therefore, we call v0 (t) = b v(t) the Rotation Equation. We have proved:

Theorem 25 (The Rotation Equation). For any given vectors b and v0 in R3 , there is one and only
one solution to the system of equations
v0 (t) = b v(t)
and
v(0) = v0 .
If b = 0, or if v0 is a multiple of b, this solution is v(t) = v0 for all t. Otherwise it is given by

(2.58).
91
This gives us another geometric way to think about what the cross product represents: Take a
non-zero vector v R3 and another non-zero vector b in R3 , and produce a curve v(t) by rotating
v through that angle kbkt about the axis b so that the rotation in the plane orthogonal to b is
counter-clockwise when viewed from along the positive axis in the direction of b. Then,
b v = v0 (0) .
That is,
v(t) = v + tb v + O(t2 ) .
One can loosely express this as saying that the cross product b v specifies the increment in v under
an infinitesimal rotation about the positive b axis.
Example 38 (Solving the Rotation Equation). Let b := (1, 4, 8) and v0 := (2, 0, 2). Let us solve
the Rotation Equation v0 (t) = b v(t) with v(0) = v0 .
First, we compute that kbk = 9. Hence
kbk = 9
and
u3 =
1
(1, 4, 8) .
9
Next, we compute v0 u3 = 2. Hence

2
2
(v0 ) = (2, 0, 2) (1, 4, 8) = (8, 4, 1) .
9
9
Therefore,
k(v0 ) k = 2
and
u1 =
1
(8, 4, 1) .
9
Finally,
1
(4, 7, 4) .
9
We now have all the information we need to evaluate the formula (2.58). We get
u2 = u3 u1 =
v(t) = 2 cos(9t)u1 + 2 sin(9t)u2 + 2u3 .

More explicitly, using our evaluation of u1 , u2 and u3 , this is
v(t) =
1
(16 cos(9t) + 8 sin(9t) + 2 , 8 cos(9t) + 14 sin(9t) + 8 , 2 cos(9t) 8 sin(9t) + 16)
9
(2.59)
The plot we have given just above Theorem 25 is of this solution for 0 t 2, except that the length
of b is reduced by a factor of 2 in the plot; the plot is easier to understand when kbk and kv0 k are
not too far apart.
As we have explained in connection with motion in a magnetic field, it is natural to think of a
solution v(t) of the Rotation Equation as a particle velocity. Since Theorem 25 gives us an explicit
formula for v(t) in terms of v0 and b, if we are also given the initial position x0 , we can integrate
v(t) to get
Z
x(t) = x0 +
v(r)dr .
0
When v(t) is given by the formula (2.58), the time dependence is very simple, and it is easy to do
the integrals explicitly.
92

The result is:
x(t)
=
=
k(v0 ) k
k(v0 ) k
x0
sin(kbkt)u1 +
(1 cos(kbkt))u2 + t(v0 )k
kbk
kbk

k(v0 ) k
k(v0 ) k
x0 +
u2 +
(sin(kbkt)u1 cos(kbkt)u2 ) + t(v0 )k ,
kbk
kbk
(2.60)
which we recognize as the parameterization of a Helix. This shows that the trajectory of a charged
particle in a constant magnetic field is a helix.
Here is a plot of the helix produced by using the solution of the rotation equation found in
Example 38 in (2.60), using x0 = 0, and also showing the initial velocity vector v0 and b:
There is another context in which this is natural: If x(s) is a curve in R3 parameterized by

arclength, then x0 (s) = T(s), and by the Frenet-Seret formulae (2.38), we then have T0 (s) = (s)
T(s).
Thus, whenever the Darboux vector is constant along a curve x(s), the unit tangent vector
T(s) satisfies
T0 (s) = T(s)
where is the constant value of the Darboux vector. Thus, Theorem 25 gives us an explicit formula
for T(s) in terms of T(0) and , making appropriate adjustments to the notation only. Then since
x0 (s) = T(s), we can use the fundamental Theorem of Calculus to find an explicit formula for x(s)
itself: As we have just seen, x(s) will be a helix.
This raises the question: When is the Darboux vector constant?
Lemma 8 (Constant curvature and torsion). Let x(t)be a thrice differentiable curve with non-zero
speed and curvature for a < t < b. Then the Darboux vector (t) is constant on the interval (a,b) if
and only if the curvature (t) and the torsion (t) are constant on (a, b).
Proof: Suppose first that the curvature and torsion are constant. Then (t) = T(t) + B(t).
93
Differentiating, and using the Frenet-Seret formulae (2.38),

0 (t)
= T0 (t) + B0 (t)
= v(t) (t) T(t) + v(t)(t) B(t)
= v(t)(t) ( T(t) + B(t))
= v(t)(t) (t) = 0 .
Thus, when the curvature and torsion are constant, so is the Darboux vector.
For the converse, suppose that the Darboux vector is constant. Then (t) = T(t), and so
0 (t) = T0 (t) = v(t)(t) N(t) = 0
since the Darboux vector is always orthogonal to N. A similar calculation shows that 0 (t) = 0.
Thus, whenever, the curvature and torsion are constant, the Darboux vector is constant, and
thus the curve x(s) is a helix. Conversely, we have already seen that for any helix, the curvature and
torsion are constant. Hence we have proved:
Theorem 26 (Curvature, torsion and helices). A thrice differentiable curve with non-zero speed and
curvature is a helix if and only if it has constant curvature and torsion.
2.2.4
Planetary motion
Consider a planet of mass m orbiting a star of mass M . According to Newtons Universal Theory of
Gravitation, the force that attracts the planet to the star has magnitude
GM m
r2
where G is the gravitational constant and r is the distance between the centers of the star and the
planet.
The center of mass will stay fixed, and since the star is generally much, much more massive than
the planet, it is an excellent approximation to regard the star as fixed. We take its position to be the
origin of our coordinate system, and denote the position of the planet at time t by x(t). Since the
gravitational force acting on the planet is directed towards the star, and hence towards 0, Newtons
second law tells us that the motion of the planet satisfies
x00 (t) =
GM
x(t) .
kx(t)k3
(2.61)
We are going to determine the orbit of the planet, which is to say, the path traced out by x(t).
The key to this, as in our study of motion in a constant magnetic field, is to find constants of the
motion. The crucial initial observation for us there was that in a constant magnetic field H, v(t) and
H v(t) are both independent of time. This gave us the plane in which v(t) was then found to rotate.
The key to easily solving (2.61) is to find constants of the motion. There are two vector-valued
constants of the motion for (2.61):
94
Definition 35 (Momentum, angular momentum and the Runge-Lenz vector). Let x(t) be a twice
differentiable curve in R3 satisfying (2.61). Then the momentum p(t), the angular momentum L(t)
and the Runge-Lenz vector A(t) of the planet are given by
p(t) = mx0 (t) = mv(t),
(2.62)
L(t) = x(t) p(t) .
(2.63)
and
A(t) = p(t) L(t) GM m2
x(t)
.
kx(t)k
(2.64)
The momentum is not a constant of the motion for this system: computing the derivative, we
find
p0 (t) = mx00 (t) =
GM m
x(t) .
kx(t)k3
(2.65)
But notice that p0 (t) is a multiple of x(t). Because of this and the fact that p(t) is a multiple of
x0 (t),
L0 (t) = x0 (t) p(t) + x(t) p0 (t) = 0 .
This shows that L is a constant of the motion, and in fact, since it is vector valued, it provides us
three scalar constants of the motion.
Next, let us compute A0 (t) in two steps. First, since L0 (t) = 0,
(p(t) L(t))0
=
=
p0 (t) L(t)
GM m
x(t) (x(t) p(t)) .

kx(t)k3
(2.66)
Using Laplaces identity u (v z) = (u z)v (u v)z from Theorem 10, we find

x (x p) = (x p)x kxk2 p .
Therefore,
(p(t) L(t))0 =
GM m
GM m
(x(t) p(t))x(t) +
p(t) .
3
kx(t)k
kx(t)k
(2.67)
Second, we compute

d
1
1
d
1
x(t) =
kx(t)k x(t) +
x0 (t)
2
dt kx(t)k
kx(t)k
dt
kx(t)k
and
d
dp
1
1
kx(t)k =
x(t) x(t) = p
x(t) x0 (t) =
x(t) x0 (t) .
dt
dt
kx(t)k
x(t) x(t)
Thus, altogether,
d
dt
GM m2
x(t)
kx(t)k

=
GM m
GM m
(x(t) p(t))x(t) +
p(t) .
kx(t)k3
kx(t)k
(2.68)
Combining (2.67), (2.68) and the definition of the Runge-Lenze vector A, we see that A0 (t) = 0.
Summarizing our conclusions, we have proved:
95
Theorem 27 (Constants of the motion for planetary orbits). Let x(t) be a solution of (2.61). Then
the angular momentum vector L(t) and the Runge-Lenz vector A(t) are both constants of the motion:
L(t) = L(0) = L
and
A(t) = A(0) = A
for all t.
We are now ready to solve for the orbits. Consider any orbit with given A and L. Notice that
L = 0 if and only if the motion of the planet is straight towards or away from the star, and of course
this does not describe an orbit. Therefore, to avoid trivialities, let us suppose that L 6= 0. Since
by definition, the angular momentum is orthogonal to x(t), the orbit lies in the plane given by the
equation
Lx=0 .
This plane is called the orbital plane. (It is, in fact, the osculating plane to the orbit at each time t.)
Next, suppose that A = 0. Then x(t) A = 0 for all t, and by then the definition of A(t),
x(t) A(t) = x(t) p(t) L(t) GM m2 kx(t)k ,
(2.69)
and so x(t) p(t) L(t) = GM m2 kx(t)k. Using the triple product identity
x(t) p(t) L(t) = L(t) (x(t) p(t)) = L(t) L(t) = kLk2 .
(2.70)
Thus, (2.69) reduces to

kLk2
.
GM m2
This means that x(t) traces out a circle around the star in the orbital plane, and with the radius
kx(t)k =
R=
kLk2
.
GM m2
(2.71)
Since the orbit is circular, the velocity x0 (t) is tangent to the circle and therefore orthogonal to
x(t). It follows that, kLk = mkx(t)kkx0 (t)k = mRkx0 (t)k and hence the speed v = kx0 (t)k is constant
and given by
kLk
.
(2.72)
mR
Thus, when A = 0, the motion is constant speed circular motion in the orbital plane. We can
v=
get a relation between v and R by eliminating kLk between (2.71) and (2.72)
GM m2 R = kLk2 = v 2 m2 R2 ,
and so
GM
.
(2.73)
R
This is usually expressed in terms of the period of the orbit, since that is what one can most
v=
easily determine directly by astronomical observation. The period of the orbit, traditionally denoted
T , is the time it takes the planet to complete one circular orbit. This is given by
T =
2R
.
v
(2.74)
96

Then from (2.74) and (2.73) we obtain
2
T =
4 2
GM
R3 ,
(2.75)
which is the usual form of Keplers Third Law in the case of circular orbits, with one very important
contribution by Newton: Keplers law stated that the square of the period was proportional to
the cube of the radius, but did not provide a formula for the proportionality constant. Newtons
derivation does, and moreover, the constant G, giving the strength of the gravitational force, can be
measured on the Earth. Then, making astronomical observations from the Earth, one can determine
the radii and the periods of the orbits of the other planets circling the Sun. Using this data in (2.75),
we can calculate M , the mass of the Sun.
In this way, one can weigh the Sun with a telescope, since the orbits of the planets are nearly
circular.
The estimates of the value of the Sun computed in this way would vary depending on which
planet is used in part because the orbits are not exactly circular. To get a better treatment, let us
now consider the case A 6= 0.
From (2.69) and (2.70), we have
x(t) A = kLk2 GM m2 kx(t)k .
(2.76)
Identifying the orbital plane with R2 in such a way that A points in the e1 direction, and and writing
x(t) = (x(t), y(t)), we can rewrite (2.76) as
GM m2
x2 (t) + y 2 (t) = kLk2 kAkx(t) .
Dividing through by GM m2 , we see that for all t, (x, y) := (x(t), y(t)) satisfies
p
x2 + y 2 = b ax
where
a=
kAk
GM m2
and
b=
(2.77)
kLk2
,
GM m2
which, after some simple algebra, leads to the conclusion that (x(t), y(t)) satisfies
(G2 M 2 m4 kAk2 )x2 + G2 M 2 m4 y 2 + 2kLk2 kAkx = kLk2
for all t. This is the equation of a conic section. It gives a closed orbit if and only if the coefficient of
x2 is strictly positive; i.e., if and only if kAk2 < G2 M 2 m4 . In this case, the orbit will be an ellipse.
In fact, we can say more: For b > 0 and 0 a < 1, the equation (2.77) is the equation of and
ellipse in the x, y plane with one focus at the origin, and the other at the point ( 2f, 0), semi-major
axis of length R+ running along the x-axis, and semi-minor axis of length R r inning along the
y-axis, where

f=
a
1 a2

b
R+ =
1
1
b and R =
b.
2
1a
1 a2
(2.78)
97
To see this, we verify that

k(x, y) ( 2f, 0)k + k(x, y) (0, 0)k = 2R+
(2.79)
if and only if x, y satisfies (2.77) with a and b related to f and R+ by the first two equalities in (2.78).
A standard geometric definition of an ellipse is that it is the set of points in the plane for which the
sum of the distances from two fixed points in the plane the two foci - is constant, and the constant
value is necessarily the length of the major axis. It is left as an exercise to prove this, though it may
well be familiar from high-school geometry. However, note that the set of points (x, y) satisfying
(2.79) is exactly the set of points such that the sum of the distances from ( 2f, 0) and (0, 0) is 2c,
in which necessarily c f , and c is the length of the semi-major axis.
Thus, all closed orbits are ellipses with the sun at one focus. In this case the magnitude of the
Runge-Lenz vector necessarily satisfies
kAk GM m2
and the direction is from the Sun out to the point on the ellipse that is closest to the Sun, which is
known as the perihelion. (The point at the other end of the major axis, which is the point farther
from the sun, is the aphelion.
Keplers First Law states that the obits of the planets are ellipses with the sun at one focus, and
we have now explained how this may be derived from Newtons Universal Theory of Gravitation. q
Kelpers Second Law states that as the planet orbits the Sun, the straight line segment connecting
the Sun and the planets sweeps out equal areas in equal times. As we shall see later on when we
discuss area, this is a direct consequence of something we have already observed, namely the fact
that L, and in particular kLk is constant.
2.2.5
The specification of curves through differential equations
We have now seen several examples in which curves x(t) are specified by giving some initial data,
such as x(0) = x0 and x0 (0) = v0 and then a formula for the acceleration x00 (t) in terms of the
position x(t) and velocity x0 (t):
x00 (t) = f (x(t), x0 (t)) .
(2.80)
Equation (2.80) is an example of a differential equation. (The terminology derivative equation might
seem more apt, since the equation (2.80) relates the second derivative x00 (t) to the first derivative
x0 (t), and to x(t). However, the terminology was coined when people thought more commonly in
terms of differentials and infinitesimals than derivatives.)
In ballistic motion without friction, the function f is simply constant: f (x, vv) = f0 . In ballistic
motion with friction, f (x, v) is a linear function of v, and is independent of x: f (x, v) = f0 v. In
the Rotation Equation, f (x, v) is a somewhat more complicated linear function of v: f (x, v) = b v.
Finally, in the case of plenty motion, we have f (x, v) independent of v, but depending in a non-linear
manner on x :
f (x, v) =
GM
x.
kxk3
98

As we have seen in each of these cases, for each given x0 and v0 , there exists a unique curve x(t)
that satisfies the equation (2.80) and also the initial conditions x(0) = x0 and x0 (0) = v0 . Later,
we shall prove general existence and uniqueness theorems telling us when a differential equation and
initial data are guaranteed to specify a unique curve, and we shall also develop methods for computing
the solution. This more general investigaion of the subject will have to wait until we have developed
more tools of multivariable calculus. However, the idea of specifying a curve through a differential
equation together with initial data is far too important to be postponed so far into the future, and
as we have seen, we can already explicitly solve a number of important cases with the tools presently
at our disposal.
2.3
Exercises
2.1 Let x(t) = (t + 1 , t2 ). This is a parameterization of the parabola y = (x 1)2 .

(a) Compute v(t) = x0 (t) and a(t) = x00 (t).
(b) Compute v(t) and T(t).
(c) Find the tangent line to this curve at t = 1.
2.2 Let x(t) = (t2 , 4/ t , t) for t > 0.

(a) Compute v(t) = x0 (t) and a(t) = x00 (t).
(c) Find the tangent line to this curve at t = 1.
2.3 Let x(t) and y(t) be two continuous curves in Rn . Show that f (t) := x(t) y(t) is a continuous
real valued function of t. Also for n = 3, show that
z(t) := x(t) y(t)
is a continuous curve in R3 .
2.4 Let x(t) = ( cos(t) , sin(t) , t/) where r > 0. The curve x(t) is a helix in R3 .
(a) Compute v(t) and a(t).
(c) Compute the curvature (t) and the torsion (t), as well as N(t) and B(t).
(d) Compute the Darboux vector (t).
(e) Find the tangent line to this curve at t = /4, and the equation of the osculating plane to the
curve at t = /2. Find the intersection of this line and plane.
2.5: Let x(t) be the curve given by
x(t) = (et cos t, et sin t, et ) .
(a) Compute the arc length s(t) as a function of t, measured from the starting point x(0), and find
an arc-length parameterization of this curve
2.3. EXERCISES
99
(b) Compute curvature (t) and torsion (t) as a function of t.

(c) Find an equation for the osculating plane at time t = 0
x(t) = (t3/2 , 3t, 6t1/2 )
for t > 0.
(a) what is the arc length along the curve between x(1) and x(4)?
(c) Find an equation for the osculating plane at t = 1, and find a parameterization of the tangent
line to the curve at t = 1.
2.7: Let x(t) be the curve given by x(t) = (t, t2 /2, t3 /3).
(a) Find the equation of the osculating plane at t = 1.
(b) Compute the distance from the origin to the osculating plane at t = 1.
x(t) = (2t, t2 , t3 /3) .
(a) Compute the arc length s(t) as a function of t, measured from the starting point x(0).
(c) Find equations for the osculating planes at time t = 0 and t = 1, and find a parameterization of
the line formed by the intersection of these planes.
2.9 Consider the ellipse in R2 given by the equation
y2
x2
+
=1
a2
b2
where a, b > 0.
(a) Show that the path traced out by the parameterized curve x(t) = (a cos(t), b sin(t)) is this ellipse.
In other words, x(t) = (a cos(t), b sin(t)) is a parameterization of this ellipse.
(b) Compute the curvature (t), and find the minimum and maximum values of curvature on the
ellipse, and the places where the curvature takes on these values.
2.10 Let x(t) be the curve given by x(t) = (t , 2 ln(t) , 1/t) for t > 0.
(a) Find the arc length along the curve from x(1) to x(3).
(b) Find the arc length along the curve from x(1) to x(t) as a function of t.
(c) Find the arc length parameterization x(s) of this curve.
2.11 Find the arc length along the parabola y = (x 1)2 from the point (0, 1) to the point (1, 0).
(See Excercise 2.1.)
100
2.12 Find the arc length parameterization of the curve given by x(t) = (t2 , 4/ t , t) for t > 0.
(See Excercise 2.2.) What is the arc length along the segment of the curve joining x(1) and x(4)?
2.13 Let b = (2, 1, 2). Let x(t) be the curve given satisfying the initial value problem
x0 (t) = b x(t)
and
x(0) = (1, 1, 1) .
(a) Compute x() and find the arc length along the curve from x(0) to x().
(b) Compute the curvature and torsion for this curve as a function of t.
2.14 Let b = (4, 7, 4). Let x(t) be the curve given satisfying the initial value problem
x0 (t) = b x(t)
and
x(0) = (2, 2, 1) .
(a) Compute x() and find the arc length along the curve from x(0) to x().
(b) Compute the curvature and torsion for this curve as a function of t.
2.15 Show that for b > 0 and 0 a < 1, the set of points (x, y) that satisfy (2.77) is an ellipse with
one focus at the origin, and the other at ( 2f, 0), and semi-major axis R+ where f and R+ are
given in terms of a and b by (2.78).
2.16 Let x(t) be the curve given by x(t) = (cos t + 1, cos t + sin t, sin t + 1).
(a) Compute curvature (t) and torsion (t) as a function of t.
(b) Find an equation for the osculating plane at time t = 0
(c) Find the distance between the plane given by x y + z = 0 to x(t) as a function of t.
2.17 Consider the helix whose Darboux vector is (3, 0, 4), with x(0) = 0, and {T(0), N(0), B(0)} =
{e1 , e2 , e3 }. Find a formula for x(s), the arc-length parameterization of the helix.
Chapter 3
CONTINUOUS FUNCTIONS
3.1
3.1.1
Continuity in several variables

Functions of several variables
Consider a function f from Rn to Rm . Such a function takes a vector variable x as input, and returns
a vector f (x) as output. For example, consider the function f from R2 to R3 given by
f (x, y) = (x2 + y 2 , xy , x2 y 2 ) .
(3.1)
(We will usually use the notation f (x) or f (x, y) instead of the more cumbersome f ((x, y)), but they
all mean the same thing.) Introducing the functions
f1 (x) = x2 + y 2
f2 (x) = xy
and
f3 (x) = x2 y 2 ,
and with f (x) defined as in (3.1), we can write f as a vector whose entries are functions from R2 to
R.
f (x) = (f1 (x) , f2 (x) , f3 (x))
Often, the questions we ask about f (x) can be answered by considering the entry functions f1 , f2
and f3 one at a time.
What kinds of questions will we be asking about such functions? Many of the questions have to
do with solving equations involving f . For example, consider the equation
f (x) = (2, 1, 0) .
(3.2)
We can rewrite this as a system of equations using the entry functions f1 f2 and f3 :
f1 (x, y)
f2 (x, y)
f3 (x, y)
0.
(3.3)

101
102
CHAPTER 3. CONTINUOUS FUNCTIONS

More explicitly,
x2 + y 2
xy
x2 y 2
0.
(3.4)
To solve a system of equations in several variables, one has to eliminate variables. In this case,
elimination is not hard: Adding the first and third equation, we find 2x2 = 2, or x2 = 1. The third
equation now tells us y 2 = x2 = 1. So x = 1 and y = 1. Going to the second equation, we we
that if x = 1, then y = 1 also, and if x = 1, then y = 1 also. Hence the equation (3.2) has exactly
two solutions:
x1 = (1, 1)
and
x2 = ( 1, 1) .
That is, for these vector x1 and x2 ,

f (x1 ) = f (x2 ) = (2, 1, 0) ,
and no other input vectors x yield the desired output.
In general, it is not easy to solve vector equations in vector variables of the form f (x) = b except
in the special case that f : Rn Rm is linear. In the linear case, as we shall see, there are very
effective algorithms for computing the solution in a finite number of operations. Sometimes, as in
our example just above, one can also do this for non-linear functions f as well.
However, it is not always possible to arrive at the solution of a non-linear equation in finitely
many steps. The way forward is to use, in principle, infinitely many steps. This is not as bad as it
may sound. There are good methods, such as Newtons method for producing a sequence of vectors
{xk } with the property that
lim xk = z
(3.5)
where z is an exact solution of the equation f (x) = b; i.e.,

f (z) = b .
(3.6)
Since human beings cannot do infinite computations, we can never carry out all of the computations
needed to arrive at z; we must stop at the kth stage for some k, and be satisfied with the approximation
xk . But this is not so bad. Even with familiar numbers like , which is one solution to the equation
sin(x) = 0, we can only compute exactly a finite number of digits in its decimal representation.
It will be the same here when we apply Newtons method to solving equations involving functions of
vector variables: We will generate a sequence of approximate solutions of rapidly improving accuracy,
and from this sequence, we can exactly compute any desired number of decimals in each of the entires
of the solution to which the sequence converges.
In this sense, methods of successive approximations yield methods of exact computation.
As soon as we contemplate methods of successive approximation for the solution of equations
like f (x) = b, we are led to the concept of continuity for functions of a vector variable. Suppose you
3.1. CONTINUITY IN SEVERAL VARIABLES
103
have a sequence {xk } of vectors in the domain of f satisfying (3.5) where z satisfies (3.6). We would
like to think of the vectors xk , at least for large k, as approximate solutions of the equation f (x) = b.
Thus we would hope that for large k, kf (xk ) bk would be very small. More precisely, we would
hope that:
lim f (xk ) = b
(3.7)
In this case, we are justified in considering {xk } as a sequence of approximate solutions of the equation
f (x) = b. However, there are functions f , even of one variable, for which (3.5) and (3.6) do not imply
(3.7).
Example 39 (Bad behavior under limits). Let f (x) be the function on R defined by
1 x > 0
f (x) =
0 x 0
Consider the sequence {xk } given by xk = 1/k. Let z = 0, Then
lim xk = z
and
f (z) = 0 ,
however
lim f (xk ) = 1 6= 0 .
The function f in this example is discontinuous: It has a jump at x = 0.

The important class of functions for which (3.5) and (3.6) do imply (3.7) is the class of continuous
functions. However, when the input variable is a vector in Rn , n 2, the notion of continuity
is somewhat more subtle than it is when the input variable is in R1 : A function on R2 can be
discontinuous without having any jumps, as we explain next.
3.1.2
Continuity in several variables
In plain words, but a bit roughly, this is what continuity at x0 means for a function f from Rn to
Rm :
We are guaranteed that f (x) f (x0 ) up to any given small margin of error provided that x x0
up to some other small enough margin of error: The output of the function will be close to f (x0 )
provided the input is sufficiently close to x0 .
Let us remove the roughness, and make this precise. There are two margins of error involved
one on the input, and one on the output. Let > 0 denote the margin of error on the input, and let
> 0 be the margin of error on the output. We are looking for a guarantee that if kx x0 k < ,
then kf (x) f (x0 )k < , and we want there to be such a guarantee with > 0 no matter how small
> 0 may be. Of course , the margin of error on the input, will depend on . But continuity means
such a guarantee is possible for all > 0.
It is not be possible to make such a guarantee for all functions. Even for the single variable
function in Example 39, such a guarantee is not possible when x0 = 0 and is any number less than
1: There exist x arbitrarily close to x0 = 0 with |f (x) f (x0 )| = 1. This function f is somewhat
hypersensitive: You change the input ever so slightly, and the response changes completely.
104
Definition 36 (Continuity). A function f from Rn to Rm is continuous at x0 in case for every > 0,

there is a () > 0 so that
kx x0 k ()
kf (x) f (x0 )k .
(3.8)
for all x in the domain of f . The function f is continuous if it is continuous at each x0 in its domain.
Whether a function is continuous or not is a matter of considerable practical importance.
If a function f from Rn to Rm is not continuous at a solution x0 of f (x) = b, it is no use at all
to find a vector x1 with even kx1 x0 k < 10300 since without continuity, there is no guarantee that
f (x1 ) is at all close to f (x0 ) = b.
Without continuity, only exact solutions are meaningful. But these will often involve irrational
numbers that cannot be exactly represented on a computer. Therefore, whether a function is continuous or not is a serious practical matter.
How do we tell if a function is continuous? The advantage of the precise mathematical definition
over the intuitive one is that it is checkable. Here is a simple example in the case in which m = 1, so
only the input is a vector:
Example 40 (Continuity of linear functions from Rn to R). Let f : Rn R be given by
f (x) = a x .
We may assume a 6= 0, or else f is constant, and hence obviously continuous.
The basis question before us, the question of continuity of f , amounts to the question: How large
is |f (x) f (x0 )| compared to kx x0 k? The Cauchy-Schwarz inequality provides the means to make
the comparison. (Comparisons are exactly what inequalities are all about.) We have:
|f (x) f (x0 )| =
It follows that
kx x0 k
|a x a x0 |
|a (x x0 )|
kakkx x0 k .
1
|f (x) f (x0 )| .
kak
Thus, with = /kak, we have the guarantee we seek, and f is continuous at x0 . Since x0 was any
vector in Rn , f is continuous on Rn .
A very important special case is that of the coordinate functions
cj (x1 , . . . , xn ) = xj .
Note that
cj (x) = ej x ,
and so the coordinate functions are continuous, as a spacial case of the example treated here.
The next theorem says that all questions about the continuity of functions from Rn to Rm reduce
to questions about continuity of functions form Rn to R.
105
Theorem 28 (Continuity and continuity of the entry functions). Let f : Rn Rm where

f (x) = (f1 (x), . . . , fm (x)) .
Then f is continuous if and only if each fj is continuous as a function from Rn to R.
Proof: Suppose that f : Rn Rm is continuous. Then for each > 0, there is a () > 0 so that
(3.8) is true. Then since for each j, |fj (x) fj (x0 )| kf (x) f (x0 )k,
kx x0 k ()
|fj (x) fj (x0 )| ,
and thus each fj is continuous.
Conversely, suppose that each fj is continuous. Then for any > 0, there is a j (/ n) so that
kx x0 k j (/ n)

|fj (x) fj (x0 )| .
n
But since
kf (x) f (x0 )k =
n
X
1/2
|fj (x) fj (x0 )|2
j=1
if we define
n max {|fj (x) fj (x0 )| } ,

j=1,...,n
() = min {1 (/ n) , . . . , n (/ n) } ,

j=1,...,n
then () > 0 and (3.8) is true, so that f is continuous.

Example 41 (Continuity of linear functions from Rn to Rm ). Let f : Rn Rm have the form
f (x) = (a1 x, . . . , am x) for some set of vectors {a1 , . . . , am } in Rn . As we shall see, the general
linear transformation from Rn to Rm has this form.
Since each of the entry functions in f is continuous, as shown in Example 40, it follows from
Theorem 28 that f is continuous.
The next theorem provides convenient means for checking continuity of functions from Rn to R.
Before presenting the theorem, let us look at an important example to which it pertains.
Example 42. Let f (x, y) be given by
f (x, y) = xy .
This function is a second order polynomial in the two variables x and y. As we shall soon see, all
polynomials, in any finite number of variables, are continuous. The reasons this is true can be readily
grasped by examining this simple example.
To show that f is continuous, pick any x0 = (x0 , y0 ) R2 . We must then compare |f (x, y)
p
f (x0 , y0 )| with kx x0 k = (x x0 )2 + (y y0 )2 . There are two variables involved, and the basic
idea is to add and subtract so that one writes f (x, y) f (x0 , y0 ) as a sum of differences in which only
one variable at a time is changing. For example,
xy x0 y0 = (x x0 )y + x0 (y y0 )
(3.9)
106
so that
|xy x0 y0 | |x x0 ||y| + |x0 ||(y y0 )| .
(3.10)
As we have observed a number of times,

|x x0 |
(x x0 )2 + (y y0 )2 = kx x0 k
and, likewise, |y y0 | kx x0 k. Therefore,

|f (x, y) f (x0 , y0 )| = |xy x0 y0 | (|x0 | + |y|)kx x0 k .
We are getting there, but there is still a factor of |y| on the right hand side, and we would like
the right hand side to be expressed only in terms of constants, such as x0 and y0 , and in terms of
kx x0 k. However, the factor of |y| is readily dealt with: By the triangle inequality and what we have
said above,
|y| = |y0 + (y y0 )| |y0 | + |y y0 | |y0 | + kx x0 k .
Putting it all together, we have
|f (x, y) f (x0 , y0 )| = |xy x0 y0 | (|x0 | + |y0 | + kx x0 k)kx x0 k .
Whenever kx x0 k 1, this means that
|f (x, y) f (x0 , y0 )| = |xy x0 y0 | (|x0 | + |y0 | + 1)kx x0 k .
Therefore, if

() := min

, 1 ,
|x0 | + |y0 | + 1
then
kx x0 k () |f (x) f (x0 )| .
This shows that f is continuous at x0 . Since x0 is any vector in R2 , f is continuous on R2 . The
same divide and conquer strategy that was used in (3.9) can be used to show that the product of
any two continuous functions f and g from Rn to R is continuous. This is part of the next theorem.
Theorem 29 (Building continuous functions from Rn to R). Let f and g be continuous functions
from some domain U in Rn to R. Define the functions f g and f + g by f g(x) = f (x)g(x) and
(f + g)(x) = f (x) + g(x). Then f g and f + g are continuous on U . furthermore, if g 6= 0 anywhere
in U , then f /g defined by (f /g)(x) = f (x)/g(x) is continuous in U . Finally, if h is a continuous
function from R to R, then the composition h f is continuous on U .
Proof: Consider the case of f + g. Fix any > 0, and any x0 in U . Since f and g are continuous
there is a f (/2) > 0 and a g (/2) > 0 so that
kx x0 k f (/2)
|f (x) f (x0 )| /2
kx x0 k g (/2)
|g(x) g(x0 )| /2
and
107
Now define
() := max{f (/2) , g (/2)} .
Then, whenever |x x0 | (),
|(f + g)(x) (f + g)(x0 )| |f (x) f (x0 )| + |g(x) g(x0 )| /2 + /2 =
so that
|x x0 | ()
|(f + g)(x) (f + g)(x0 )| .
This proves the continuity of f + g. The other cases are similar. In fact, the proof for products
very closely follows the treatment of the special case in Example 42, and the proof for composition
is exactly like the proof in the corresponding single variable theorem. Finally, composition with
h(t) = 1/t allows one to reduce the analysis of disvion to that of multiplication. Therefore, the
proofs for these remaining cases are left as exercises.
Example 43 (Continuity piece by piece). To apply Theorem 29, try to recognize a function as being
built out of known continuous pieces. For example, consider
z(x, y) = cos (1 + x2 + y 2 )1
This is built out of the continuous building blocks

f (x, y) = x
g(x, y) = y
and
h(x, y) = 1 .
Indeed,

z(x) = cos
h
h + f f + gg

(x) .
Repeated application of Theorem 29 then shows z(x) is continuous.

Example 44 (Continuity of polynomials and rational functions). A polynomial in several variables
is a sum of monomials, which a functions of the form
f (x1 , . . . , xn ) = a
n
Y
xj j
j=1
where each pj is a non-negative integer, and a is a constant. For example, on R3 with coordinates x,
y, and z,
3x2 y 3 z
and
2x4 z 2
are monomials. Since the coordinate functions are continuous, as shown in Example 40, and since
products of continuous functions are continuous by Theorem 29, it follows that monomials are continuous. Then, by Theorem 29 once more, and sum of finitely many monomials is continuous, and
hence every polynomial, e.g.,
f (x, y, z) = 3x2 y 3 z 2x4 z 2
is continuous.
108

A rational function is a ratio of polynomials; e.g.
f (x, y, z) =
3x2 y 3 z + x yz
.
1 + x4 y 2
By Theorem 29 once more, a rational function is continuous at all points x0 where the denominator
is not zero. In the example at hand, the denominator is never zero, and the function is continuous
on all of R3 .
3.1.3
Continuity and limits
When we wish to determine whether a given function is continuous or not, one way is to make direct
use of the and definition. This is a good strategy for many problems, and it is what we have done
until now.
The main theorem of this subsection gives us an alternative to the the and analysis of
continuity. It provides a characterization of continuity in terms of limits: A function is continuous if
and only if one can pass limits to the inside of the function.
Theorem 30 (Characterization of continuity in terms of limits). Let f : Rn Rm . Then f is
continuous at x0 Rn if and only whenever {xk } is a sequence in Rn with
lim xk = x0 ,
then
lim f (xk ) = f (x0 ) .
Proof: Suppose that f is continuous, and that limk xk = x0 . We must show that limk f (xk ) =
f (x0 ).
For this purpose, pick > 0. Since f is continuous, there exists () > 0 so that kf (x)f (x0 )k
whenever kxx0 k < (). Since limk xk = x0 , there is an N() so that kxk x0 k () whenever
k N() . Therefore,
k N()
kf (xk ) f (x0 )k .
This shows that limk f (xk ) = f (x0 ).

On the other hand, suppose that f is not continuous at x0 , Then for some > 0, there is no
() > 0 such that every point x with kx x0 k satisfies kf (x) f (x0 )k < . In particular, for
each natural number k, there exists xk such that kxk x0 k < 1/k but kf (x) f (x0 )k . Then
limk xk = x0 , but {f (xk )} cannot possibly converge to f (x0 ).
Example 45 (Anaysis of continuity via limits, 1). Let f (x, y) be given by
xy
(x, y) 6= (0, 0)
2
2
f (x, y) = x + y
0
(x, y) = (0, 0) .
Is f continuous?
109
To solve this problem, note that away from (0, 0) f is a continuous rational function. Hence, the
only question is whether it is continuous at 0 = (0, 0). The function is continuous if and only if for
every sequence {xn } with limn xn = 0, it is the case that limn f (xn ) = f (0) = 0.
Let us try some sequences {xn } that approach 0, starting with some obvious ones. Let the
sequence approach 0 from along the y-axis, taking xn = (1/n, 0). In this case, we get f (xn ) = 0 for
all n, so certainly limn f (xn ) = 0. Since the function is symmetric in x and y, the same thing
happens if we take the sequence to approach 0 from along the y-axis; i.e., taking xn = (0, 1/n). But
if we take the the sequence to approach 0 from along the line y = x; that is with xn = (1/n, 1/n), we
find
f (1/n, 1/n) =
1
1/n2
=
2
2
1/n + 1/n
2
for all n, and hence for this choice of {xn }, limn xn = 0, but
lim f (xn ) =
1
6= 0 = f (0) .
2
Hence, this function is discontinuous.

In general, s soon as we have found one sequence for which limn xn = x0 , but limn f (xn ) 6=
f (x0 ), the analysis is over: The function is definitetely not continuous at x0 .
Example 46 (Anaysis of continuity via limits, 2). Let f (x, y) be given by
x y
(x, y) 6= (0, 0)
f (x, y) = x4 + y 2
0
(x, y) = (0, 0) .
Is f continuous?
To solve this problem, note that away from (0, 0) f is a continuous rational function. Hence, the
only question is whether it is continuous at 0 = (0, 0). The function is continuous if and only if for
every sequence {xn } with limn xn = 0, it is the case that limn f (xn ) = f (0) = 0.
Let us try some obvious sequences {xn } that approach 0, starting with the choices
xn = (1/n, 0)
xn = (0, 1/n)
and
xn = (1/n, 1/n) .
For each of these choices, you will find limn f (xn ) = f (0) = 0.
So, let us try somethng else: Let us choose the sequence to approach 0 along the line y = ax with
a 6= 0. Every line through the origin that is not one of the axes has this form, and weve already tried
the axes. So let us now try
xn = (1/n, a/n) .
We find
f (1/n, a/n) =
a/n3
a/n
=
1/n4 + a2 /n2
1/n2 + a2
for all n, and hence also for this choice of {xn }, limn xn = 0 = f (0).
So, we have looked at all the ways of approaching the origin along lines, and we still have not
found an example of discontinuity. This, however, does not prove continuity. Proving continuity
110
requires us to consider all sequences that approach the orgin, no matter how they do it, spiraling in,
perhaps, turning cartwheels along the way.
In fact, this function is discontinuous. To see this, consider the sequence
xn = (1/n, 1/n2 ) ,
which approaches the origin along the parabola y = x2 . We find
f (1/n, 1/n2 ) =
1
1/n4
=
+ 1/n4
2
1/n4
for all n, and hence for this choice of {xn }, limn f (xn ) = 1/2 6= f (0) = 0. Thus, this function is
shown to be discontinuous at 0.
In the last two examples, we have shown that functions were discontinuous by finding a sequence
{xn } that converges to 0, but for which {f (xn )} does not converge to f (0). To show that a function
is not continuous, you only need to dsiplay one such sequence.
On the other hand, to prove that a function is continuous at x0 , you must show that for every
sequence {xn } that converges to x0 , it is always the case that {f (xn )} converges to f (x0 ). We have
even seen that it is not enough to look only at sequences that approach x0 from along all lines; it can
be that problems only show up for sequences that approach x0 along some curve, as in Example, 46
or perhaps even in a more complicated way.
With so many possibilities to deal with, how can we ever show that a function is continuous in
this manner? The squeeze principle provides a way forward.
3.1.4
The squeeze principle is several variables
As in the single variable calculus, we can use the squeeze principle to make comparisons with simple
functions.
Lemma 9 (Squeeze principle). Let f be a given real valued function defined on some open set U
about x0 Rn . Suppose there exists a function g from [0, ) to [0, ) such that limt0 g(t) = 0 and
such that
|f (x) f (x0 )| g(kx x0 k) .
(3.11)
Then for all sequences {xn } such that limn xn = x0 , limn f (xn ) = f (x0 ), and hence f is
continuous at x0 .
Proof: Pick any > 0, Then since limt0 g(t) = 0, there is an r > 0 so that g(t) < for all t < r .
Consider any sequence {xn } in U (where f is defined) such that limn xn = x0 . Then there
is an Nr so that for all n Nr , kxn x0 k < r . But then by (3.11), for all n Nr ,
|f (xn ) f (x0 )| g(r ) < .
Since > 0 is arbitrary, this proves that limn f (xn ) = f (x0 ). Since the convergence sequence was
arbitrary, this proves that f is continuous at x0 .
111
Example 47 (Anaysis of continuity via the squeeze principle, 1). Let f (x, y) be given by
f (x, y) =
x2 + y 2
x2 + y 2 + 1 1
(x, y) 6= (0, 0)
(x, y) = (0, 0) .
Is f continuous?
To solve this problem, note that away from (0, 0), f is a continuous function. Hence, the only
question is whether it is continuous at 0.
To apply the squeeze principle, we need to relate f (x) to a function g of kxk recall that in this
case x0 = 0. But this is easy: For x 6= 0, since f (0) = 2,
f (x) f (0) = p
kxk2
kxk2 + 1 1
2 .
Hence if we define
g(t) :=
t2
2
t2 + 1 1
for t 0, we have
|f (x) f (0)| = g(kxk) = g(kx 0k) .
Thus with this choice of g, (3.11) is even true as an equality. Hence by Lemma 9, f is continuous
provided limt0 g(t) = 0. Using lHospitals Rule, one readily checks that
lim
t0
t
=2.
t+11
Hence, the function f is continuous.

The nice thing about the function f in the last example is that f only depends on on x through
kxk. That it, it only sees the magnitude of x, and not the direction. As far as f is concerned, all
sequences that approach the origin are on the same footing: Since it does not see the direction, it
does not notice any difference between sequences that approach the origin along a coordinate axis,
or along a spiral, or a parabola, or anything else.
Of course, not every function is so nice. But, now, how does one find the function g with which
to put on the squeeze? The following inequalities are often helpful in this regard.
Lemma 10 (Sums and powers). For all numbers a, b 0,
21p (a + b)p ap + bp (a + b)p
for all
p1
(3.12)
0<p1
(3.13)
and
21p (a + b)p ap + bp (a + b)p
Also
ab
and the is equality in (3.14) if and only if a = b.
a2 + b2
2
for all
(3.14)
112
Proof: Assume that at least one of a and b is not zero, since otherwise everything is trivial. Then
since a/(a + b) and b/(a + b) are both numbers in the unit interval, and since for p > 1, tp < t for t
in the unit interval,
1=
b
a
+
a+b a+b
a
a+b
p

+
b
a+b
p
=
a p + bp
.
(a + b)p
Thus proves that ap + bp (a + b)p , and one sees that the inequality reverses if instead 0 < p 1.
Next, since the function (t) = tp is convex for p 1 (and concave for 0 < p 1), we have for
p 1 that

a+b
2
p
ap + bp
.
2
This shows that 21p (a + b)p ap + bp , and again the inequality reverse for 0 < p 1. This proves
(3.12).
Next,
0
a2 + b2
(a b)2
=
ab ,
2
2
which proves the last part.

Before turning to our next example, let us explain how (3.14) motivates the choice of the sequence
(1/n, 1/n2 ) that we used in Example 46.
If you are looking for a sequence {xn } that converges to 0, but for which {f (xn )} does not go to
zero, it is a good idea to look for a sequence along which the denominator (not the numerator) goes
to zero especially fast. In this case, the denominator is x4 + y 2 . By (3.14),
x4 + y 2 2x2 y
with equality if and only if x2 = y. This suggests that the denominators will be particularly small
along the sequence (1/n, 1/n2 ).
Example 48 (Anaysis of continuity via the squeeze principle, 2). Let f (x, y) be given by
2xy
(x, y) 6= (0, 0)
r
r
f (x, y) = |x| + |y|
0
(x, y) = (0, 0)
where r > 0. For which values of r is f continuous at 0 = (0, 0)?
Let us try the obvious sequences
xn = (1/n, 0)
xn = (0, 1/n)
and
xn = (1/n, 1/n) .
We find
f (1/n, 0) = f (0, 1/n) = 0
and
f (1/n, 1/n) =
2/n2
= nr2 .
1/nr
Only the third one is interesting. We see that for r = 2, f (1/n, 1/n) = 1 for all n, so that
limn f (1/n, 1/n) = 1 6= f (0, 0). Even worse, for r > 2, limn f (1/n, 1/n) = . Thus, for
r 2, f is not continuous at the origin.
113
However, while this suggests that for r < 2, f might be continuous at the origin, it does not prove
that it is as we have seen, things might go wrong with other sequences. Still, if you try a few more,
you will see that nothing goes wrong. It is then time to try to use the squeeze principle.
The goal is to compare f with a function g that only depends on x through kxk. We are looking
to have
f (x) g(kxk) .
Let us take r < 2, and start with the denominator. We would like to estimate the denominator in
terms of x2 + y 2 = kxk2 , and we would like to estimate the denominator by something smaller, so as
to get something larger than f . The denominator is
|x|r + |y|r = (x2 )r/2 + (y 2 )r/2 (x2 + y 2 )r/2 = kxkr ,
where we have used the right-side inequality in (3.13).
Next, let us work on the numerator. We would like to estimate the numerator in terms of
x2 + y 2 = kxk2 , and wed like to estimate the absolute value of the numerator in terms of something
larger, so as to get something larger than f . The absolute value of the numerator is
2|x||y| x2 + y 2 = kxk2 ,
where we have used (3.14).
Now let us put it all together: For x 6= 0, and r < 2,
|f (x)| =
kxk2
2|xy|
= kxk2r .
|x|r + |y|r
kxkr
Hence we have (3.11) with x0 = 0 and g(t) = t2r . Thus by Lemma 9, f is continuous at the origin
for 0 < r < 2.
3.1.5
Continuity versus separate continuity
Many questions about the behavior of a function f : Rn R can be answered by examining single
variable slices of the function. We now explain this fundamental strategy in some detail.
Given any function f : Rn R, and any parameterized line x0 + tv in Rn , define the function
g : R R by
g(t) = f (x0 + tv) .
The function g is a garden variety single variable function, and it describes a slice of the function
f . Studying all such slices, we can learn many useful things about f .
For example, consider the function f (x, y) given by
f (x, y) =
3(1 + x)2 + xy 3 + y 2
.
1 + x2 + y 2
Here is a plot of the graph of z = f (x, y) for 3 x, y 3:
114
6
4
2
0
1
0
1
x
1
2
2
3
Here is another picture of the same graph, but from a different angle that give more of a side
view:
6
2
3
0
x
In both graphs, the curves drawn on the surface show points that have the same z value, which
we can think of as representing altitude. Drawing them in helps us get a good visual understanding
of the landscape in the graph. They are called contour curves, and are drawn in on any topographic
map. (The formula for f (x, y) was chosen to produce a graph that looks like a bit of a mountain
landscape.)
Now that we understand what the graph of z = f (x, y) looks like, lets slice it along a line.
Suppose for example that you are walking on a path in this landscape, and that the projection of
your path on the surface down into the x, y plane is the line x(t) = x0 + tv with
x0 = (1, 1)
and
v = (1, 1) .
Consider the vertical plane over this line; i.e., the plane in R3 that contain this line and the z-axis.
Here is a picture of this vertical plane slicing through the graph of z = f (x, y):
115
2
3 2 1 0
3
1 2 3 2 1 0 1 2
The next graph shows the altitude profile as we walk along the graph; this curve is where the
surface intersects our vertical plane:
7
6
5
4
3
2
1
3
Compare the last two graphs, and make sure you see how the curve in the second one corresponds
to the intersection of the plane and the surface in the first one.
In this example, we have illustrated the slicing strategy with a function f : R2 R only so we
could plot graphs and produce visual aids to our understanding. But the basic formula
g(t) = f (x0 + tv)
can be used in any number of dimensions.
The slicing strategy turns out to be a very useful method for studying functions f : Rn R, as
we shall see. However it has its limitations. For instance, one might hope that a function f : Rn R
is continuous if and only if for each choice of x0 and v in Rn , the function g(t) = f (x0 + tv) is
continuous. This is not the case.
In fact, we have already seen this in Example 46. While the function f in Example 46 is
discontinuous at the origin, its slice along every line through the origin (and therefore every line,
since the only discontinuity is at the origin) is continuous. You could walk along the landscape
116
described by f on any straight line path through the origin, but you would never run into a cliff.
Otherwise put, there are no jumps in any linear slice.
Here is an even more dramatic example.
Example 49 (Discontinuity without jumps). Define f : R2 R by
2x4 y
1
(x, y) 6= (0, 0)
f (x, y) = x2 + y 2 x8 + y 2
0
(x, y) = (0, 0) .
Since f is the product of rational functions whose denominators vanish only at (0, 0), f itself is
continuous away from (0, 0). Hence the slice of f along any line that does not pass through the origin
is continuous. We will now show that the slice of f along every line is continuous.
By what we have said above, it remains to consider lines through the origin. Notice that f
is identically equal to zero along the y-axis. (There is a factor of x in the numerator.) Constant
functions are certainly continuous, so this slice of f is continuous.
Now consider any other line through the origin. This has the equation y = ax for some a R,
a 6= 0. On the line y = ax, for x 6= 0,
f (x, ax) =
2x5 a
1
2xa
1
=
2
2
2
8
2
2
2
6
x +a x x +a x
1 + a x + a2
which is a continuous and even differentiable function of x. Therefore, for each non-zero a R,
the function g(t) defined by
g(t) := f (0 + t(1, a))
is continuous. Hence, the slice of f along every line through 0 is continuous.
However, f itself is not continuous. To see this, consider the sequence {xn } given by xn :=
(1/n , 1/n4 ). Evidently, limn xn = 0, while for each n,
f (xn ) =
1 2n8
n2
.
=
2n2 2n8
2
Hence limn f (xn ) = , and so not only is f discontinuous, there are points arbitrarily close to
the origin at which f takes on arbitrarily large positive values.
If you consider the sequence xn := (1/n , 1/n4 ), you can see that there are points arbitrarily
close to the origin at which f takes on arbitrarily large negative values.
Yet as you walk along the slice over any straight line through the origin, you never encounter
any jumps or an singularities of any sort.
At this point you might wonder how wise we have been in choosing our definition of continuity.
We are excluding functions f that vary continuously on the line segment connecting any two points
from the class of continuous functions. Are we not being too restrictive?
Let us consider an alternate definition:
Definition 37 (Separate continuity). A function f (x, y) on R2 is separately continuous in case
for each x0 , the function y 7 f (x0 , y) is a continuous function of y, and if for y0 , the function
x 7 f (x, y0 ) is a continuous function of x.
3.2. CONTINUITY, COMPACTNESS AND MAXIMIZERS
117
The function f in Example 49 is separately continuous. Moreover, separate continuity is easier

to check than continuity it can be done one variable at a time. Could it be that the n dimensional
analog of this is actually a better generalization of continuity to several variables? Unfortunately, no.
Mathematical definitions are made the way they are because of what can be done with them.
They are discarded unless they capture concepts that are useful in problem solving.
Part of the problem-solving value of the concept of continuity lies in its relevance to minimum
maximum problems. You know from the theory of functions of a single variable that if g is any
continuous function of x, then g attains its maximum on any closed interval [a, b]. That is, there is
a point x0 with a x0 b so that for every x with a x b,
g(x) g(x0 ) .
In this case, we say that x0 is a maximizer of g on [a, b]. Finding maximizers is one of the important
applications of the differential calculus.
In the next section, we show that continuity is the right hypothesis for proving a multi-variable
version of this important theorem, and that separate continuity is not enough. Separate continuity
is easier to check, but alas, it is just not that useful.
3.2
3.2.1
Continuity, compactness and maximizers

Open and closed sets in Rn
In this subsection we introduce the class of subsets of Rn that generalizes the class of bounded, closed
intervals in R.
Definition 38 (Open ball of radius r, and bounded sets). For each x Rn , and each r > 0, the
open ball of radius r about x is the subset Br (x0 ) of Rn defined consisting of all y Rn such that
kx yk < r. That is,
Br (x) = {y Rn : kx yk < r} .
A set A Rn is bounded in case A Br (0) for some r > 0.
Definition 39 (Open and closed sets). A set C Rn is closed in case whenever {xk } is a sequence
of points belonging to C, and the limit z = limk xk exists, then z C. A set U Rn is open in
case whenever x U , there is an r > 0 so that Br (x) U . The empty set is defined to be both
open and closed.
It is a good exercise to use the triangle inequality to show that for each x Rn and each r > 0,
Br (x) is open. You may be wondering how one could every prove a set with infinitely many points
is closed. Are there not infinitely many sequences to be dealt with in the proof? Yes, and so we had
better do that implicitly. The shall prove a theorem that is useful in this regard. First, we state a
useful definition.
118
Definition 40 (Level sets). Given any function f : Rn R, and any a R, the set consisting of
all solutions of the equation f (x) = a is called the level set of f at height a. It is written as
{ x : f (x) = a } .
Likewise, the sub-level set of f at height a and the super-level set of f at height a, respectively, are
the sets
{ x : f (x) a }
and
{ x : f (x) a } .
Example 50 (Spheres as level sets). Consider the function f (x) = kxk2 =
Pn
j=1
x2j . For each r > 0,
the sphere of radius r is the the level set of f through r2 .

Theorem 31 (Continuity and closed level sets). Let f : Rn R be continuous. Then for each
a R, the level set of f through a is closed, as are the sub-level set of f through a and the super-level
set of f through a.
Proof: Consider the level set { x : f (x) = a }. If this is empty, we are finished, since the empty set
is closed by definition. Otherwise, consider any convergent sequence {xk } such that each xk belongs
to the level set. Since {xk } is convergent, there exists z Rn with limk xk = z. Since f is
continuous, we have
lim f (xk ) = f (z) .
But since each xk lies in the level set, f (xk ) = a for all k. Thus f (z) = a, and hence the limit point
z also belongs to the level set. Since the convergent sequence was an arbitrary convergent sequence
in the level set, this proves that the level set is closed.
The proofs for sub-level sets and super-level sets are very much the same, and are left to the
reader.
Example 51 (Spheres are closed). The function f (x) = kxk2 =
Pn
j=1
x2j is continuous. Indeed,
by Example 40, each of the coordinate functions; i.e, the functions sending x to xj , j = 1, . . . , n
Pn
are continuous. Then by Theorem 29, the function sending x to j=1 x2j is continuous. Thus, by
Theorem 31, for each r > 0, the sphere of radius r is closed.
We conclude this subsection with one more relation between open and closed sets that shall be
useful to us later on.
Definition 41 (Complementary sets). For any set A Rn , the complement of A, Ac , is the subset
of all x Rn with x
/ A.
Theorem 32 (Complementarity of open and closed sets). (1) Let U Rn be open. Then U c , the
complement of U , is closed. (2) Let C Rn be closed. Then C c , the complement of C, is open.
Proof: Let U Rn be open, and consider any convergent sequence {xk } in U c . We must show that
z = limk xk U c .
119
Suppose not. Then z = limk xk U . Since U is open, for some r > 0, Br (z) U . But since
z = limk xk , for all sufficiently large k, kxk zk < r, and this means that all such xk belong to
U . But this is impossible, since each xk U c . Hence, z = limk xk U c .
Next, let C Rn be closed. We must show that for each x C c , there is some r > 0, Br (x) C c .
Suppose not. Then for each k N, there is some xk B1/k (x) C. By construction, for all > 0
k > 1/
kxk xk < .
Thus, limk xk = x. Since C is closed and since each xk belongs to C, it would have to be the
case that x C. However, this is impossible since x C c . Therefore, there is some r > 0 with
Br (x) C c .
3.2.2
Minimizers and maximizers
Definition 42 (Minimizers and maximizers). Let f be a function defined on a closed set C Rn

with values in R. Then x0 C is a maximizer of f in C is case
f (x) f (x0 )
for all
xC .
(3.15)
xC .
(3.16)
Likewise, Then x0 C is a minimizer of f in C is case

f (x) f (x0 )
for all
Notice that x0 is a minimizer of f in C if and only if x0 is a maximizer of f in C. Therefore, it

suffices to prove theorems on the existence of maximizers: Each such theorem implies a corresponding
theorem for minimizers.
We shall show in the next subsection that every continuous real valued function defined on a
bounded closed set C Rn has a maximizer in C. First, we show by example that this is not true if
one replace continuity by separate continuity.
Example 52 (Separately continuous, but no maximizer). Consider the function f from Example 49.
As we have seen there, this functions is separately continuous, and even more, its slice along every line
is continuous. Let C = {(x, y) x2 + y 2 1 } which is plainly bounded, and is closed by Theorem 31.
We will now show that f has neither a maximizer nor a minimizer on C, and in fact, is unbounded
above and below on C, all despite being separately continuous. Indeed, we have already done the work.
We have seen in Example 49 that
f (1/n , 1/n4 ) =
n2
2
and
f (1/n , 1/n4 ) =
n2
.
2
For n > 1 both (1/n, 1/n4 ) and (1/n, 1/n4 ) belong to C. Hence f is unbounded above and below
on C.
However, for continuous functions of several variables, there is an analog of the single variable
theorem. This alone makes continuity a much more useful concept than separate continuity.
120
3.2.3
Compactness and existence of maximizers
Definition 43 (Compact sets). A set C Rn is compact in case it is closed and bounded.

They key to the existence of maximizers, and many other problems as well, is the following
theorem which says that a set is compact if and only if every infinite sequence of points in the set has
a subsequence that converges to a point in C. It is one of the fundamental theorems of mathematical
analysis. The theorem is powerful because often it is easy to check that a set is compact, and then
you know something significant about every infinite sequence in that set. The very interesting proof
relies on the pigeonhole principle and the completeness of the real numbers.
Theorem 33 (The Bolzano-Weierstrass Theorem). Let C be a compact subset of Rn . Then for every
sequence {xn } in C, there is a subseuence {xnk } and a z C such that
lim xnk = z .
On the other hand, if C is not compact, then there exists a sequence {xn } in C that has no
convergent subsequence.
Proof: We will prove this for n = 2 so that we can draw pictures. Once you understand the idea for
n = 2, you will see that it applies for all n.
Since C is a closed and bounded set, there are numbers xc , yc and r so that C is contained in
the square
xc x xc + r
and
yc y yc + r .
The shaded region is the closed, bounded set C.

Now consider any infinite sequence {xn } of points in C. To obtain a convergent subsequence,
first divide the square into four congruent squares. Since the four squares cover C, by the pigeonhole
principle, at least one of the four squares must be such that it contains infinitely many terms of the
sequences {xn }.
121
Here the upper left square contained infinelty many terms and we chose it.
Now define xn1 to be the first term in {xn } that belongs to the chosen square.
Next, subdivide the square in the first step into four smaller squares as in the diagram below.
Again, by the pigeonhole principle, one of these must be such that it contains infinitely many terms
of the sequence {xn }. Choose such a square.
Here we chose the lower left square in the previously selected square.
Now define xn2 to be the first term in {xn } after xn1 that belongs to the chosen square.
Iterating this procedure produces a sequence of points {xnk } such that for each m > 0, the
sequence {xnk : k m} ies in a square of side length r2m . This follows from the fact that the
procedure described above produces a nested set of squares.
122
Since the side length is reduced by a factor of 2 with each subdivision, and since it starts at r/2,
at the mth stage we have a square of side length 2m r. Thus, for all k, ` m, xnk and xm` belong
to a square of side length 2m r. How far apart can they possibly be? No further than the length of
the diagonal of the square, which is 2 times 2k r; i.e., 21/2m r. Thus, for any > 0, choose m so
that 21/2m r < , and then for this m,
k, ` m
k xnk xn` k < .
Since for any coordinate index j, |(x` )j (xm )j | kx` xk k,

k, ` m
|(xnk )j (xn` )j | < .
This means that {(xnk )j }, where j is fixed and k indexes the terms of the sequence, is a Cauchy
sequence. By the completeness property of the real numbers, or what is the same thing, the construction of the real numbers out of the rational numbers, every Cauchy sequence has a limit, and so
there exists a number zj R so that
lim (xnk )j = zj .
(3.17)
Now define a vector z by (z)j = zj for j = 1, . . . , n. Then (3.17) implies

lim xnk = z .
Then, since C is closed, and {xnk } is in C, z belongs to C.

Our first application of the Bolzano-Weierstrass Theorem is to the existence of maximizers.
Theorem 34 (Continuity and Maximizers). Let f be a continuous function defined on a compact
set C Rn with values in R. Then there is point z in C so that
f (x) f (z)
for all
xC .
(3.18)
Proof: Let B be the least upper bound of f on C. That is, B is either infinity if f is unbounded on
C, or else it is the least number that is greater than or equal to f (x) for all x C. We aim to show
that B is finite, and that there is an z C with f (z) = B. Then z is the maximizer we seek.
First, suppose that B = . Then for each n, there is an xn C such that f (xn ) n. By
Theorem 33, there is a convergent subsequence {xnk } with limit z C. Since f is continuous,
f (z) = lim f (xnk ) .
k
123
However, since f (xnk ) nk k for all k, limk f (xnk ) does not exist. This contradiction shows
that B must be finite.
Now, by the definition of the Least Upper Bound, for each n, the set
{ x C : f (x) B 1/n }
is not empty. Therefore, for each n we may choose xn in this set. By the squeeze principle, since
B 1/n f (xn ) B, limn f (xn ) = B.
By Theorem 33, the sequence {xn } has a convergent subsequence {xnk } with limit z C. Since
f is continuous, f (z) = limk f (xnk ) = B. Thus, z C and f (z) = B f (x) for all x C.
For the final part, suppose C is not compact. Then either C is not closed, or not bounded, or
both. If it is not closed, there is some sequence {xn } in C that converges to some z
/ C. In this
case, every subsequence of {xn } converges to z
/ C, and so no subsequence of {xn } can converge to
an element of C. Likewise if C is unbounded, there is a sequence {xn } in C with kxn k n for all n,
and evidently no subsequence of this sequence converges at all. Thus, when C is not compact, there
are sequences in C for which no subsequence converges to an element of C.
The Bolzano-Weierstrass Theorem has consequences, as we shall see throughout this course. We
close this section by explaining one concerning isometries:
Definition 44. Let f be a function from U Rn to Rm with the property that for all x, y U ,
kf (x) f (y)k = kx yk .
Then f is called anisometry. That is f is an isometry in case it preserves the distances between points;
the distance between the images equals the distance between the original points.
Observe that isometries are always continuous. It is worth writing down a proof with explicit
reference to the definitions involved.
Theorem 35 (Isometries on compact sets are invertible). Let C be a compact set in Rn . Let f be
an isometry defined on C with values in C. Then f is an invertible function from C onto C.
Proof: First, f is one-to-one: If for some x, y C, f (x) = f (y), then
0 = kf (x) f (y)k = kx yk ,
and so y = x.
Showing that for each y C there is an x C such that f (x) = y takes more effort; we shall
use Theorems 33 and 34 for this.
Suppose there is some y C such that y 6= f (x) for any x C. The function
g(x) := kf (x) yk
is a continuous function on C, being the composition of continuous functions. Thus, by Theorem 34,
there is a z C with g(z) g(x) for all x C. Since y 6= f (z), g(z) := r > 0. and thus there is
some r > 0 such that
kf (x) yk r
(3.19)
124
for all x C. Now, inductively define the sequence {xn } by defining x1 = y, and then
xn+1 = f (xn )
for all n 1.
By Theorem 33, there exists a subsequence {xnk } converging to some z C. Since every
convergent sequence is a Cauchy sequence, there are k and ` so that k < ` and
kxnk xn` k < r/2 .
But with f (m) denoting the mth composition power of f ,
xnk = f (nk ) (y)
xn` = f (n` ) (y) .
and
By the isometry property,

r/2 kxnk xn` k = kf (nk ) (y) f (n` ) (y)k = ky f (n` nk ) (y)k .
Now define x := f (n` nk 1) (y) so that
f (n` nk ) (y) = f (x) .
Then ky f (x)k < r/2 which contradicts (3.19). This contradiction shows that there cannot exist
any y C such that y 6= f (x) for all x C. Hence, f maps C onto C. Since we have already seen
that f is one-to-one, this proves f is invertible.
We can now use Theorem 35 to give a short proof of the fundamental Lemma 1, which says that
there does not exist any set of n + 1 orthonormal vectors in Rn . Indeed, the unit sphere S, consisting
of all unit vectors in Rn is compact, as we have seen in this section.
Second Proof of Lemma 1: Given any set {u1 , . . . , un } of n orthonormal vectors in Rn , define a
function f from Rn to Rn by
f (x1 , . . . , xn ) =
n
X
xj uj .
j=1
Then, since {u1 , . . . , un } is orthonormal,

n
2
n
X

X

2
kf (x) f (y)k = (xj yj )uj =
|xj yj |2 = kx yk2 .

=1
j=1
Thus, f is an isometry. Likewise, we see kf (x)k = kxk so that if x S, then also f (x) S. Thus,
the restriction of f to S is an isometry from S into S.
Now consider any other unit vector un+1 . By Theorem 35, f not only maps S into S, it maps S
onto S, and since un+1 S, there is an x = (x1 , . . . , xn ) S such that f (x) = un+1 ; i.e.,
un+1 =
n
X
xj uj .
j=1
Taking the dot product of both sides with un+1 , 1 = un+1 un+1 =
Pn
j=1
xj (un+1 uj ). If un+1 were
orthogonal to each uj , the right hand side would be zero, and this is impossible. Hence there does
not exist any set of n + 1 orthonormal vectors in Rn .
3.3. EXERCISES
125
There is a strong contrast between the two proofs we have given of Lemma 1. The first proof
involved the explicit construction of a product of Householder reflections taking {u1 , . . . , un } into
{e1 , . . . , en }. The second proof avoids this, and is much shorter but this is because it makes use
of the Bolzano-Weierstrass Theorem. Both kinds of proof have their place. Sometimes an explicit
construction will not be possible, and an argument making use of the Bolzano-Weierstrass Theorem
is the only way to proceed. However, when an explicit construction is possible, it is often worhtwhile
to make it: The explicit construction used in our first proof of Lemma 1 is what enabled us to prove
that an orthonormal basis {u1 , u2 , u3 } of R3 is right handed if and only if it is related to {ve1 , e2 , e3 }
by a rotation.
3.3
Exercises
3.1 Complete the proof of Theorem 29.

3.2 A function f defined domain U Rn with values in R is called a Lipschitz continuous function
in case there is some number M so that
kf (x) f (y)k M kx yk
(3.20)
for all x and y in U .

(a) Show that a Lipschitz continuous function is continuous by finding a valid margin of error on the
input; i.e., a valid () that has a very simple form: proportional to .
(b) For R > 0, let U denote the ball of radius R about the origin; i.e., U = BR (0). Let f (x) be
defined on U by f (x) = kxk2 . Using the identity
kxk2 kyk2 = (x y) (x + y)
and the Cauchy-Schwarz inequality, show that f is Lipschitz on U with Lipschitz constant 2R.
(c) Let f : Rn Rm have the form f (x) = (a1 x, . . . , am x) for some set of vectors {a1 , . . . , am }
in Rn , as in Example 41. Show that is Lipschitz continuous on Rn .
3.3 Consider the function f defined by
f (x1 , x2 ) = sin(x1 ) cos(x2 ) .
Note that
f (x1 , x2 ) f (y1 , y2 ) = (sin(x1 ) sin(y1 )) cos(x2 ) + sin(y1 )(cos(x2 ) cos(y2 ))
(3.21)
Show that
| sin(x1 ) sin(y1 )| |x1 y1 |
and
| cos(x2 ) cos(y2 )| |x2 y2 | .
(This is a single variable problem, and the fundamental theorem of calculus can be applied). Combine
this with the identity (3.21) to show that f satisfies (3.20) with M = 2.
126
3.4 Let f (x, y) be given by

2
x sin(xy)
x6 + y 2
f (x, y) =
(x, y) 6= (0, 0)
(x, y) = (0, 0) .
(a) For any a, b R, define the sequence {xn } by xn = (a/n, b/n) . Compute lim f (xn ).
n
(b) For any a, b R, define the sequence {xn } by xn = (a/n, b/n3 ) . Compute lim f (xn ).
n
(c) Is the function f continuous? justify your answer.

3.5 Consider the function defined by
(x + y) ln(x2 + y 2 ) (x, y) 6= (0, 0)

f (x, y) =
0
(x, y) = (0, 0).
Is this function is continuous? Justify your answer.
x y
2
f (x, y) = x + y 4
(x, y) 6= (0, 0)
(x, y) = (0, 0).
Is this function is continuous? Justify your answer. Hint: You make a ratio larger and simpler at
the same time if you discard a positive term for the denominator.
|x|r sin(y/x) x 6= 0
f (x, y) =
0
x=0
where r > 0. For which values of r, if any, is f continuous?
3.8 Let a and b be given vectors in R3 such that neither is a multiple of the other. Define a function
f : R3 R by
f (x) = a (b x) .
Define
x0 =
1
ab .
ka bk
Show that
f (x) f (x0 )
for all unit vectors x R3 . In other words, show that x0 is the maximizer of f on the unit sphere in
R3 .
3.9 Given m vectors {a1 , . . . , am } in Rn , define the function f from Rn to R by
f (x) =
m
X
j=1
(aj x)2 .
3.3. EXERCISES
127
Show that f has a both a maximizer and a minimizer on the closed unit ball
Pn
2
B = {x :
j=1 xj 1}, but has only a minimizer, and no maximizer, on the open unit ball
Pn
2
B = {x :
j=1 xj < 1}. Hint: Show that the maximizer on B lies on the boundary of B.
3.10 Let f be any function from Rn to Rm . For any set A Rm , define f 1 (A) to be the set of all
points x, if any, in Rn such that f (x) A. The set f 1 (A), which may be the empty set, is called
the preimage of A under f . Do not be misled by the notation: f 1 (A) is defined whether or not the
function f itself is invertible.
(a) Prove that f is continuous if and only if whenever A is an open set in Rm , then f 1 (A) is an
open set in Rn . This result provides a way to talk about continuity without explicitly bringing and
into the discussion. It also has other uses:
(b) Use the result of part (a) to give a short proof that whenever f is a continuous function from
Rn to Rm , and g is a continuous function from Rm to R` , then g f is a continuous function from
Rn to R` .
3.11 Let K Rn be compact. Show that for each x Rn , there exists at least one point yK (x) K
(depending on x, as indicated in the notation) such that
kx yK (x)k kx zk
for all z K. Define the distance from x to K to be the number kx yK (x)k for this choice of y.
The function
dK (x) := kx yK (x)k
then gives the distance from x to K. Show that the function dK is a continuous function on Rn . Is
it Lipschitz continuous?
3.12 Let K Rn be compact, and let f be a continuous function from Rn to Rm . Define L Rm by
L := {y Rm : y = f (x) for some x K } .
Is L necessarily compact? Justify your answer.
128
Chapter 4
DIFFERENTIABLE FUNCTIONS
4.1
4.1.1
Vertical slices and directional derivatives

Directional derivatives and partial derivatives
We now pick up with the slicing idea introduced in Chapter 3, and try to take derivatives along
slices of a function f : Rn R in order to understand the nature of the graph of xn+1 = f (x) in
Rn+1 . Of course, we can only actually plot the graph for n 2, so we will be especially interested in
the case n = 2 at the beginning.
Definition 45 (Directional derivatives). Given a function f (x) defined in an open subset U of Rn ,
and some point x0 Rn , and also a non zero vector v in Rn , the directional derivative of f at x0 in
the direction v is defined by
f (x0 + hv) f (x0 )
,
h
provided this limit exists. If the limit does not exist, the directional derivative does not exist.
lim
h0
(4.1)
Given f , x0 and v, the function

g(t) = f (x0 + tv)
(4.2)
represents the slice of f over the line x0 + tv in Rn . Then the directional derivative of f at x0
in the direction v is just g 0 (0). This means that directional derivatives can be computed by familiar
single variable methods.
Example 53 (Slicing a function along a line). For example, if f (x, y) =
xy 2
, x0 = (1, 1)
1 + x2 + y 2
and v = (1, 2), then

g(t) = f (1 + t, 1 + 2t) =
(1 + t)(1 + 2t)2
1 + 5t + 8t2 + 4t3
=
.
2
2
1 + (1 + t) + (1 + 2t)
3 + 6t + 5t2
The result is a familiar garden variety function of a single variable t. It is a laborious but straightforward task to now compute that g 0 (0) = 1. Please do the calculation; you will then appreciate the
better way of computing directional derivatives that we shall soon explain!

129
130
CHAPTER 4. DIFFERENTIABLE FUNCTIONS

Directional derivatives may exist for some directions, but not others:
Example 54 (Sometimes there are directional derivatives only in certain directions). Let f be the
function defined by
2
2
x y
f (x, y) = x2 + y 2
if (x, y) 6= (0, 0)
(4.3)
if (x, y) = (0, 0)
Let x0 = (0, 0) and let v = (a, b) for some numbers a and b. The question we now ask is: For which
values of a and b does there exists the directional derivative of f at x0 in direction v?
To answer this, lets compute f (x0 + hv) f (x0 ), divide by h, and try to take the limit h 0.
We find that
a2 b2
.
a2 + b2
(For v 6= 0, as in Definition 45, we do not divide by zero on the right.) Therefore

f (x0 + hv) f (x0 )
1 a2 b2
=
.
h
h a2 + b2
f (x0 ) = 0
and
f (x0 + hv) =
As h 0, this blows up, unless a = b, in which case the the right hand side is zero for every
h 6= 0, and so the limit does exist, and is zero. Therefore, for this bad function, the directional
derivative exists if and only if the direction vector v is is a non-zero multiple of either (1, 1) or
( 1, 1).
There are two special direction to consider the direction along the coordinate axes. These
special directional derivatives are called partial derivatives:
Definition 46 (Partial derivatives). Given a function f : Rn R defined in a neighborhood of x0 ,
for each 1 j n, the partial derivative of f with respect to xj at x0 is denoted by

f (x0 ) and
xj
is defined by
f (x0 + hej ) f (x0 )

(4.4)
f (x0 ) = lim
h0
xj
h
provided that the limit exists.
Partial derivatives are special cases of directional derivatives the cases in which the direction vector
v is one of the standard basis vectors ei .
Now, let us see how to compute partial derivatives and directional derivatives: This turns out to
be easy! If g(x) is related to f (x, y) through
g(x) = f (x, y0 ) ,
then
g(x0 + h) g(x0 )
f (x0 , y0 ) = lim
= g 0 (x0 ) .
(4.5)
h0
x
h
This is great news: We will not need to make explicit use of the definition of partial derivatives very
often to compute them. When computing a partial derivative, just treat all of the other variables
as constants, and differentiate in the single active variable in the usual way. Thus, we can use
everything we know about computing derivatives for functions of a single variable when we are
computing partial derivatives.
4.1. VERTICAL SLICES AND DIRECTIONAL DERIVATIVES
131
p
Example 55 (Differentiating in one variable). Let f (x, y) = 1 + x2 + y 2 and (x0 , y0 ) = (1, 2).
Then with g(x) defined by g(x) = f (x, y0 ), g(x) = 5 + x2 . By a simple computation,

g 0 (x) =
and in particular
x
5 + x2
1
f (1, 2) = .
x
6
In single variable calculus, the derivative function g 0 (x) is the function assigning the output
g 0 (x0 ) to the input x0 .
In the same way, we let

f (x, y) denote the function of the two variables x and y that assigns the
x
output
f (x0 , y0 ) to the input (x0 , y0 ). The same conventions apply to other other functions
x
and more variables.
p
Example 56 (Computing partial derivative functions). Let f (x, y) = 1 + x2 + y 2 . Holding y fixed
as a parameter instead of a variable we differentiate with respect to x as in the single variable
calculus, and find
x
f (x, y) = p
.
2
x
x + y2
Likewise,
y
f (x, y) = p
.
2
y
x + y2
Once more, because computing partial derivatives is just a matter of differentiating with respect
to one chosen variable, everything we know about differentiating with respect to one variable can be
applied in particular the chain rule and the product rule.
Example 57 (Using the single variable chain rule). The function f (x, y) =
p
1 + x2 + y 2 that we
considered in Example 56 can be written as a composition f (x, y) = g(h(x, y)) where

g(z) =
Since
z+1
1
g 0 (z) =
2 1+z
and
and
h(x, y) = x2 + y 2 .
h(x, y) = 2x ,
x
we have
1
x
f (x, y) =
g(h(x, y)) = g 0 (h(x, y)) h(x, y) = p
2x = p
,
x
x
x
2 1 + h(x, y)
x2 + y 2
as before.
What we saw in Example 57 is a generally useful fact about partial derivatives: If g is a differentiable function of a single variable, and h is a function of two (or more) variables with h/x
defined, then
g(h(x, y)) = g 0 (h(x, y)) h(x, y) ,

x
x
and similarly with y and any other variables. The validity of this identity need not be formulated as
a theorem, and does not need a new proof: It is true because of the single variable chain rule, and
because we are differentiating in the single variable x.
132
In short, as far as computing partial derivatives goes, there is nothing much new: Just pay attention
to one variable at a time, and differentiate with respect to it as usual.
Now let us see how to compute directional derivatives in terms of partial derivatives. The key is
another chain rule, which is a genuinely multivariable chain rule.
4.1.2
The gradient and a chain rule for functions of a vector variable
We have already seen in our study of continuity that knowing the behavior of slices of a function
f along lines does not tell us the whole story about the behavior of the f : We need to look at the
behavior along more general families of curves. It is the same with differentiability.
In this subsection we prove a chain rule for functions of a vector variable that is useful for understanding the behavior of f over slices along differentiable curves. That is, let x(t) be a differentiable
vector valued function in Rn . Let f be a function from Rn to R. Consider the composite function
g(t) defined by
g(t) = f (x(t)) .
Here we ask the question:
Under what conditions on f is g differentiable, and can we compute g 0 (t) in terms of x0 (t) and the
partial derivatives of f ?
Before answering this question, we make a useful definition. We organize the partial derivatives
of f into a vector. This definition will figure in most of what we do in the rest of this chapter.
Definition 47 (Gradient). Let f : Rn R have each of its partial derivatives well defined at x0 .
Then the gradient of f at x0 is the vector f (x0 ) Rn given by

f (x0 ) , . . . ,
f (x0 ) .
f (x0 ) =
x1
xn
Since you know how to compute partial derivatives, you know how to compute gradients: It is
just a matter of organizing the partial derivatives, once you have computed them, into a vector. Here
is an example:
Example 58 (Computing a gradient). With f (x) =
xy 2
, we compute that
1 + x2 + y 2
y 2 (1 + y 2 x2 )
f (x) =
x
(1 + x2 + y 2 )2
and
2xy(1 + x2 )
f (x) =
.
y
(1 + x2 + y 2 )2
Therefore,
f (x) =
1
(y 2 (1 + y 2 x2 ) , 2xy(1 + x2 ) ) .
(1 + x2 + y 2 )2
We are now ready to state our multivariable chain rule:
133
Theorem 36 (The chain rule for functions from Rn to R). Let f be any function defined in an
open set U of Rn with values in R. Suppose that each of the partial derivatives of f is defined and
continuous at every point of U . Let x(t) be a differentiable function from R to Rn . Then, for all
values of t so that x(t) lies in U ,
lim
h0
f (x(t + h)) f (x(t))

= x0 (t) f (x(t)) .
h
(4.6)
This is an important theorem, and before proving it, we make some remarks. First, the chain
rule in Theorem 36 applies to the composition of functions from R to Rn and then from Rn to R.
Both functions involved in this composition are multivariable functions on one end or the other. The
chain rule in Example 57 applies to the composition of functions from Rn to R and then from R
to R. In the latter case, as we have seen, we are really only using the single variable chain rule.
But the chain rule of Theorem 36 describes rates of change when all of the variables x1 , . . . , xn are
changing at once. In proving it, we will make essential use of the assumption of continuity of the
partial derivatives. Without this assumption, the theorem would not be true.
Second, Theorem 36 has a simple corollary that gives us a formula for computing directional
derivatives in terms of partial derivatives.
Corollary 2 (Directional derivatives and gradients). Let f be any function defined on an open set U
of Rn with values in R. Suppose that each partial derivative of f is defined and continuous at every
point of U . Then for any x0 in U , and any direction vector v in Rn ,
lim
h0
f (x0 + hv) f (x0 )

= v f (x0 ) .
h
(4.7)
Proof: Simply consider the case in which x(t) = x0 + tv, and apply Theorem 36.
If you worked through all the calculations in Example 53, you know that computing directional
derivatives straight from the definition as we did there can be pretty laborious. The good news is
that Corollary 2 provides a much better way!
xy 2
, x0 = (1, 1) and
1 + x2 + y 2
v = (1, 2) as in Example 53. In that example, we computed (the hard way) that the corresponding
Example 59 (Directional derivatives via gradients). Consider f (x) =

directional derivative is 1.
But now, from Example 58, we have that
f (x) =
1
(y 2 (1 + y 2 x2 ) , 2xy(1 + x2 )) ,
(1 + x2 + y 2 )2
and hence, substituting x = 1 and y = 1, we have f (x0 ) =

the directional derivative, as we found before with more labor.
1
(1, 4). Therefore, v f (x0 ) = 1 is
9
The reason that we did not already introduce a special notation for the directional derivative of
f at x0 in the direction v is that Corollary 2 provides one, namely v f (x0 ). We couldnt use it
in the last subsection because we hadnt yet defined gradients, but now that we have, this will be
134
our standard notation for directional derivatives, at least when we are dealing with nice functions
whose partial derivatives are continuous.
Corollary 2 provides an efficient means for computing directional derivatives: Once you have
computed the gradient, you can take the dot product with many different direction vectors and
compute many different directional derivatives without doing any more serious work. In the approach
used in Example 53, you would have to start from scratch each time you considered a new direction
vector.
We are finally ready for the proof of Theorem 36. The key to the proof, in which we finally
explain the importance of continuity for the partial derivatives is the Mean Value Theorem from
single variable calculus:
The Mean Value Theorem says that if g(s) has a continuous first derivative g 0 (s), then for any
numbers a and b, with a < b, there is a value of c in between; i.e., with a < c < b
g(b) g(a)
= g 0 (c) .
ba
The principle expressed here is the one by which the police know that if you drove 100 miles in one
hour, then at some point on your trip, you were driving at exactly 100 miles per hour.
Proof of Theorem 36: We give the proof for n = 2 to keep the notation simple. Once this case is
understood, the general case will be clear.
Fix some t, and some h > 0. To simplify the notation, define the numbers x0 , y0 , x1 and y1 by
(x0 , y0 ) = x(t)
and
(x1 , y1 ) = x(t + h) .
Using this notation, note that

f (x(t + h)) f (x(t))
= f (x1 , y1 ) f (x0 , y0 )

= f (x1 , y1 ) f (x0 , y1 ) + f (x0 , y1 ) f (x0 , y0 ) .
In going from the first line to the second, we have subtracted and added back in the quantity f (x0 , y1 ),
and grouped the terms in brackets. Why add and subtract the same thing? The point is that in the
first group, only the x variable is varying, and in the second group, only the y variable is varying.
Thus, we can use single variable methods on these groups.
To do this for the first group, define the function g(s) by
g(s) = f (x0 + s(x1 x0 ), y1 ) .
Notice that
g(1) g(0) = f (x1 , y1 ) f (x0 , y1 ) .
Then, if g is continuously differentiable, the Mean Value Theorem tells us that
f (x1 , y1 ) f (x0 , y1 ) =
for some c between 0 and 1.
g(1) g(0)
= g 0 (c)
10
135
But by the definition of g(s), we can compute g 0 (s) by taking a partial derivative of f , since as
s varies, only the x component of the input to f is varied. Thus,
g 0 (s) =
f (x0 + s(x1 x0 ), y1 )(x1 x0 ) .

x
Therefore, for some c between 0 and 1,

f (x1 , y1 ) f (x0 , y1 ) =
f (x0 + c(x1 x0 ), y1 ) (x1 x0 ) .

x
In the exact same way, we deduce that for some c between 0 and 1,

f (x0 , y1 ) f (x0 , y0 ) =
f (x0 , y0 + c(y1 y0 )) (y1 y0 ) .
y
Therefore,
f (x(t + h)) f (x(t))
h
x1 x0
=
f (x0 + c(x1 x0 ), y1 )
x
h

y1 y0
+
f (x0 , y0 + c(y1 y0 ))
.
y
h
Up to now, h has been fixed. But having derived this identity, it is now easy to analyze the limit
h 0.
First, as h 0, x1 x0 and y1 y0 . Therefore,
lim
h0
f (x0 + c(x1 x0 ), y1 ) =
f (x0 , y0 ) =
f (x(t)) ,
x
x
x
and
f (x0 , y0 + c(y1 y0 ) =
f (x0 , y0 ) =
f (x(t)) .
h0 y
y
y
lim
Also, since x(t) is differentiable

x1 x0
x(t + h) x(t)
= lim
= x0 (t)
h0
h0
h
h
lim
and
y(t + h) y(t)
y1 y0
= lim
= y 0 (t)
h0
h
h
Since the limit of a product is the product of the limits,

f (x(t + h)) f (x(t))

0
lim
=
f (x(t)) x (t) +
f (x(t)) y 0 (t)
h0
h
x
y
= f (x(t)) x0 (t) .
lim
h0
This is what we had to show.
4.1.3
The geometric meaning of the gradient
The gradient of a function is a vector. As such, it has a length, and a direction. To understand the
gradient in geometric terms, let us try to understand what the length and direction are telling us.
136

The key to this is the formula
a b = kakkbk cos .
(4.8)
Now pick any point x0 and any unit vector u in Rn . Suppose f : Rn R has continuous partial
derivatives at x0 , and consider the directional derivative of f at x0 in the direction u. By Theorem
1, this is u f (x0 ).
By 4.8 and the fact that u is a unit vector (i.e., a pure direction vector),
u f (x0 ) = kf (x0 )k cos
where is the angle between f (x0 ) and u. (This is defined as long as f (x0 ) 6= 0, in which case
the right hand side is zero.)
As u ranges over the set of unit vectors in Rn , i.e., the n 1 dimensional unit sphere in Rn , cos
varies between 1 and 1, and hence
kf (x0 )k u f (x0 ) kf (x0 )k
Recall that by Theorem 1, u f (x0 ) is the slope at x0 of the slice of the graph z = f (x) that
you get when slicing along x0 + tu. Hence we can rephrase this as
kf (x0 )k [slope of a slice at x0 ] kf (x0 )k
That is,
The magnitude of the gradient, kf (x0 )k tells us the minimum and maximum values of the slopes
of all slices of z = f (x) through x0 .
The slope has the maximal value, kf (x0 )k, exactly when = 0; i.e., when u and f (x0 ) point
in the same direction. In other words:
The gradient of f at x0 points in the direction of steepest increase of f at x0
For the same reasons, we get the steepest negative slope by taking u to point in the direction of
f (x0 ).
xy 2
, x0 = (1, 1) and let x0 = (0, 1).
1 + x2 + y 2
If z = f (x) denotes the altitude at x, and you stood at x0 , and spilled a glass of water, which way
Example 60 (Which way the water runs). Let f (x) =
would the water run?

For purposes of this question, lets say that the direction of the positive x axis is due East, and
the direction of the positive y axis is due North.
But now, from Example 58, we have that
f (x) =
1
(y 2 (1 + y 2 x2 ) , 2xy(1 + x2 )) ,
(1 + x2 + y 2 )2
and hence, substituting x = 0 and y = 1, we have

f (x0 ) =
1
(2, 0) .
4
Thus, the gradient points due East. This is the straight uphill direction. The water will run in the
straight downhill direction, which is opposite. That is the water will run due West.
4.1.4
137
Critical points
Theorem 36 has important application to minimization and maximization problems, which are problems in which we look for minimum and maximum values of f , and the inputs x that produce them.
Indeed, you see that:
If f (x0 ) 6= 0, then there is an uphill direction and a downhill direction at x0 .
If it is possible to move uphill from x0 , then f (x0 ) cannot possibly be a maximum value of
f . Likewise, if it is possible to move downhill from x0 , then f (x0 ) cannot possibly be a minimum
value of f .
If we are looking for either minimum values of f or maximum values of f in some open set U ,
and f has continuous partial derivatives everywhere in U , then it suffices to look among only at those
points x at which f (x) = 0.
Notice that it is vitally important that the set U be open. In an open set, starting from any
point, you can move around, at least a little bit, in all directions while staying inside the set. Hence
you can always move at least a little bit in any uphill or downhill direction. However, consider a set
that is not open, and has boundary points. At a boundary point, the uphill direction, say, might
take you out of the set. So the boundary point still might be a maximum. Consider for example the
function f (x, y) = x2 + y 2 defined on the closed unit disc. Each of the boundary points ( cos , sin )
maximizes f on this set, even though the gradient is non-zero at each of them: The uphill direction
specified by the gradient leads outside the set, and the fact that one can go further uphill upon
leaving the set is irrelevant when we seek to maximize the function f in the set.
The discussion so far leads us to the definition of a critical point:
Definition 48 (Critical point). Suppose that f is defined and has all of its partial derivatives continuous in a neighborhood of some point x0 . Then x0 is a critical point of f in case f (x0 ) = 0.
Example 61 (Computing critical points). Let f (x, y) = x4 + y 4 + 4xy. We readily compute
f (x, y) = 4(x3 + y, y 3 + x) .
We find that f (x, y) = (0, 0) if and only if x9 = x and y = x3 . Thus if x = 0, y = 0, If x 6= 0,
x8 = 1, and so x = 1, and y = x. Thus, there are the three critical points, namely
(0, 0)
( 1, 1)
and
(1, 1) .
The three critical points found in Example 61 are the only points at which f can possibly take
on either a maximum value or a minimum value. Computing the values of f at these critical points,
we find:
f (0, 0) = 0
and f (1, 1) = f (1, 1) = 2 .
One might be tempted to conclude from this calculation that the maximum value of f on R2
is 0, and the minimum value of f on R2 is 2. The answer is only half-right, and the reasoning is
138
wrong: On the basis of what has been worked out so far, we do not know that either a minimum or
a maximum exist.
Indeed, as you can easily check,
lim f (n, n) = .
This means that f has no maximum on R2 . On the other hand, f does have a minimum value, and
it is 2. It requirse a bit more thinking to see this:
Example 62 (Findiing a minimum). Let us show that f (x, y) = x4 + y 4 + 4xy does have a minimum
value, and it is 2. First observe that when kxk is large, f (x) is also large, so there is no point in
looking outside a compact set for the minimum. To make this precise, let us compare f with a function
of kxk, as we did in connection with the squeeze principle in Chapter Two. Using the inequalities
introduced there, we have
x4 + y 4
(x2 + y 2 )2
2
and
2|xy| x2 + y 2 .
Thus,
f (x)
kxk2
1
2kxk = (kxk 2)2 2 .
2
2
In particular,
kxk 4
f (x) 0 .
(4.9)
The set C := {(x, y) : kxk 4 } is closed and bounded. Therefore, by one of the key theorems
of Chapter 3, f has a minimum and maximum on C. While the maximum may (and in fact, does) lie
on the boundary of C, the minimum does not. Indeed, by (4.9), f (x) 0 for any x on the boundary
of C, and since f (1, 1) = f (1, 1) = 2, no such point can be a minimizer of f on C. Hence, the
minimizers all lie in the interior of C, and must be critical points of f .
Hence, there are critical points of f that are minimizers of f . From our computation of critical
points made above, we see that ( 1, 1) and (1, 1) are minimizers of f , and the minimum value of
f is 2.
Later in the course, we shall return to minimization and maximization problems, and study them
in considerable detail. As you see from this example, finding critical points is one important part of
finding maxima and minima, but only one part.
4.1.5
The gradient and tangent planes
Let g be a differentiable function on R. Then the graph of y = g(x) is a curve in R2 , and

y = g(x0 ) + g 0 (x0 )(x x0 )
(4.10)
is the equation of the tangent line to this curve at x0 . This is the line that best fits the graph
y = g(x) at x0
Now consider a differentiable function f from R2 to R. The graph of z = f (x, y) is a surface.
We now show that when f is differentiable, there is a unique plane, the tangent plane that best fits
the graph z = f (x) at x0 . Again, the derivative of f tells us what the tangent plane is.
139
For example here is the graph of z = x2 + y 2 , together with the tangent plane to this graph at
the point (1, 1).
15
10
5
0
5
10
3
2 1
15
10
5
0
5
10
3 2 1
We now ask:
How does one compute that equation of the tangent plane to the graph of a differentiable function
f from R2 to R at x0 R2 ? For that matter, what is the precise definition of this tangent plane?
Let f be such that all of its partial derivatives are continuous in an open set U R2 . Fix any
x0 U . Then for some r > 0, U contains every point in the ball of radius r centered on x0 . Hence,
whenever kx x0 k < r, x0 + t(x x0 ) U for all 0 t 1. That is, U contains the line segment
connecting x0 with x.
Then by Theorem ?? and the fundamental Theorem of Calculus,
Z
f (x) f (x0 )
=
0
d
f (x0 + t(x vx0 ))dt
dt
f (x0 + t(x x0 )) (x x0 )dt
(4.11)
Now when x x0 is small, x0 + t(x x0 )) is close to x0 for all t [0, 1]

Let us suppose that x is very close to x0 , so that kx x0 k is very small. .Then, if we make the
140
approximation x0 + t(x x0 ) x0 , then by the continuity of the partial derivatives of f ,

f (x0 + t(x x0 )) f (x0 )
for all t [0, 1]. But the right hand side is independent of t, and so if we use this approximation in
(4.11), the integral in t is trivial, and we get
1
f (x0 ) (x x0 )dt = f (x0 ) (x x0 ) .
f (x) f (x0 )
(4.12)
As we shall se below, this is a very good approximation near x0 ; it is the tangent plane approximation.
Using the approximation
f (x) f (x0 ) + f (x0 ) (x x0 ) ,
the graph of z = f (x) is approximated near x0 by the graph of
z = f (x0 ) + f (x0 ) (x x0 ) .
To better appreciate the simplicity of this, let us write x = (x, y), f (x0 ) := (a, b) and d :=
f (x0 ) f (x0 ) x0 . Then this becomes z = ax + by + d, or equivalently,
ax + by z = d .
This is the equation of a non-vertical plane in R3 .
Definition 49 (Tangent plane). Let f be a function on U R2 such that all of its partial derivatives
are continuous on U . Let x0 U . Then the tangent plane to the graph of z = f (x, y) at x0 = (x0 , y0 )
is given by
z = f (x0 ) + f (x0 ) (x x0 ) .
(4.13)
By what we have noted above, if we define

A =
f (x0 , y0 ) ,
f (x0 , y0 ) , 1
x
y
X0 = (x0 , y0 , f (x0 , y0 ))
X
(x, y, z) ,
the equation (4.13) is equivalent to

A (X X0 ) = 0 .
Notice how the gradient of f at x0 determines, but is not equal to, the normal vector A: The gradient
is a vector in R2 , and the normal vector is a vector in R3 .
Example 63 (Tangent planes). Consider the function f (x, y) = x2 + y 2 with x0 = 1 and y0 = 1.
Since f is a polynomial, its partial derivatives are continuous everywhere. Then f (x0 , y0 ) = 2 and
f (x0 , y0 ) = (2, 2). Therefore (4.13) becomes
z = 2 + (2, 2) (x 1, y 1) = 2x + 2y 2 .
141
Thus the tangent plane to the graph of f at x0 is given by

z = 2x + 2y 2 .
Here is a three dimensional graph of f and together with this tangent plane for the region
|x 1| 1
and
|y 1| < 1 :
As you see, the graphs are almost indistinguishable for x and y in the region
|x 1| 0.2
and
|y 1| < 0.2 .
Let us zoom in on this region:
The vertical separation between the graphs is getting to be a pretty small percentage of the displayed distances. The graphs are almost indistinguishable. Lets zoom in by a factor of 10, and have
a look for
|x 1| 0.02
and
|y 1| < 0.02 .
142
Now, the graphs really are indistinguishable. The geometric meaning of the differentiability of f
at x0 is exactly this good fit between the graph of z = f (x, y) and z = f (x0 , y0 ) + (x x0 , y y0 )
f (x0 , y0 ).
In our definition of the tangent plane, we have included the requirement that all of the partial
derivatives of f be continuous. This is what guaranteed that the tangent plane in the previous
example had the very close fit to the graph of the original function f that we saw in the last example.
As we see in the next example, without the continuity hypothesis, they may be no plane at all that
fits so well, and hence no plane at all deserving to be called a tangent plane.
Example 64 (No tangent plane). Consider the function f defined by
2xy
if (x, y) 6= (0, 0)
4
4
f (x, y) = x + y
0
if (x, y) = (0, 0)
(4.14)
As you can check, both partial derivatives are defined everywhere and
f (0, 0) = 0
f
(0, 0) = 0
x
and
f
(0, 0) = 0 .
y
Hence, if we naively applied that tangent plane approximation formula without first checking
continuity, we would conclude that
z=0
is a food approximation to z = f (x, y) near (0, 0). In fact, it is a terrible approximation. For instance
f (t, t) = t2
which gets larger and larger as (t, t) approaches (0, 0). The more you zoom in, the worse the approximation looks.
Moreover, no other plane does any better. If we try the approximation f (x, y) ax + by + d for
any a, b abd d, we have
|f (t, t) [at + bt + d]| = |t2 at bt d|
and this approaches infinity as (t, t) approaches (0, 0). The problem is that the partial derivatives of
f are no continuous st (0, 0).
143
We have just seen that if the partial derivative of f are not continuous, the tangent plane
approximation may not be at all meaningful. We now show that whenever the partial derivatives are
continuous, then it is a very good approximation, getting better and better the more one zooms in
as in Example 63.
Theorem 37 (Validity of the tangent plane approximation). Let f be a function on U R2 such
that all of its partial derivatives are continuous on U . Let x0 U . Then the vertical separation
between the graphs of z = f (x) and z = f (x0 ) + f (x0 ) (x x0 ); that is,

f (x) [f (x0 ) + f (x0 ) (x x0 )
is an arbitrarily small fraction of the horizontal separation kx x0 k for all x sufficiently close to x0 .
In other words, for every > 0, there is a > 0 so that
kx x0 k <

f (x) [f (x0 ) + f (x0 ) (x x0 ) < kx x0 k .
Before proving Theorem 37, let us consider carefully what it says: Refer back to the pictures in
Example 63, and make sure you see how as kx x0 k gets small, not only does the vertical separation
get small as well it becomes a small fraction of the the already small quantity kx x0 k. You should
convince yourself that if it only became small, but not a small fraction, the graphs would not become
indistinguishable as we zoom in.
Proof of Theorem 37: Fix an > 0. From (4.11) and (4.12), we have
Z

1
f (x) [f (x0 ) + f (x0 ) (x x0 )] = [f (x0 + t(x x0 )) f (x0 )] (x x0 )dt .

(4.15)
By the Cauchy-Schwarz inequality,

f (x0 + t(x x0 )) f (x0 )] (x x0 ) kf (x0 + t(x x0 )) f (x0 )]kkx x0 k .
(4.16)
Consider the function g(y) defined by

g(y) := kf (x0 + y) f (x0 )] ,
(4.17)
and note that since the partial derivatives of f are continuous, g is continuous and g(0) = 0. Thus
for all > 0, there is a > 0 such that
kyk <
g(y) < .
(4.18)
Now take y := t(x x0 ) and note that when t [0, 1] and kvx x0 k < , kyk = tkx x0 k < ,
and so going back to (4.16), we have
kx x0 k <

f (x0 + t(x x0 )) f (x0 )] (x x0 ) < .
Then using (4.17) and (4.18) in (4.16), we have

kx x0 k <

f (x) [f (x0 ) + f (x0 ) (x x0 )] < kx x0 k .
144
Finally, using this in (4.15), we conclude the proof.

What makes the tangent plane approximation so useful is that it provides a good approximation
to the possibly complicated function f (x) by the linear function ax + by + d where (a, b) = f (x0 )
and d = f (x0 ) f (x0 ) x0 . The function ax + by + d is simple because up to the constant d, it is
linear in both of the variables x and y.
They key idea of the differential calculus is approximate non-linear functions by their best linear
approximation wherever possible. The notion of differentiability shall be defined so that a function
is differentiable at a point if and only if that function has a unique best linear approximation at
that point.
Let us briefly consider single variable differential calculus from this point of view. For a function
g(t) of a single variable t, the tangent line to the graph of g at t0 is the line that best fits the graph
y = g(t) at t0 . It is the line that passes through the point (t0 , g(t0 )) with the same slope as the graph
of y = g(t) at t0 . Hence, it is given by the formula
y = g(t0 ) + g 0 (t0 )(t t0 ) .
Since the graph of y = g(t) and the tangent line to this graph at t0 pass through the exact same
point with the exact same slope, if you zoom in really close around a picture of both of them as
they pass through (t0 , g(t0 )), you will not be able to distinguish between the two of them.
Example 65 (How well tangent lines fit). Consider g(t) = t3 2t + 2, and t0 = 1. The tangent line
to the graph of y = g(x) at t0 = 1 is the graph of
y = g(1) + g 0 (1)(t 1) .
Since g(1) = 1 and g 0 (1) = 1, the tangent line is the graph of
y = 1 + (t 1) = t .
Here are three graphs of y = g(t) that are zoomed in closer and closer about the point (1, 1):
In the first graph, the t values range from 0.85 to 1.15. In the second, they range from 0.95 to
1.05. In the third, they range from 0.995 to 1.005. In the third graph, the curve and the tangent line
are almost indistinguishable.
There is only one line that fits so well: For a line of any other slope, the two graphs would not
even look parallel, let alone the same, when we zoom in. Also, they must both pass through the
4.2. LINEAR FUNCTIONS FROM RN TO RM
145
point (t0 , g(t0 )) to have any sort of fit at all. Since the point and the slope determine the line, there
is only one line that fits this well.
For functions f from R2 to R that have continuous partial derivatives, it is the tangent plane
that provides the best linear approximation we shall soon explain in what sense it is best . But
what about functions f from Rn to Rm ? To move forward, we need to do three things:
(1) We need to explain what a linear function from Rn to Rm is.
(2) We need to explain what best linear approximation means in precise terms.
(3) We need to explain why it is so useful to approximate non-linear functions by linear functions,
and how methods of exact calculation can be based on a sequence of successive linear approximations.
In the next section we deal with the first task, and prepare the way for the third task.
4.2
Linear functions from Rn to Rm
We begin with a definition:

Definition 50 (Linear functions). Let f be a function defined on Rn with values in Rm . Then f is
a linear function in case for all s, t R and all x, y Rn .
f (sx + ty) = sf (x) + tf (y) .
(4.19)
Specializing to y = 0, we see that for all s R and x Rn ,

f (sx) = sf (x) .
A function f : Rn Rm with this property is said to be homogeneous. Thus, linear functions are
always homogeneous.
Further specializing to s = 0, since 0x = 0 for all x, (4.19) becomes
f (0) = 0 .
Thus, a linear function always has the output value 0 at the input value 0. The following theorem
establishes another key property of linear functions
Theorem 38 (Linear functions respect linear combination). Let f : Rn Rm be linear. Then for
r
X
any linear combination
xj xj in Rn ,
j=1
r
r
X
X
f
xj x j =
xj f (xj ) .
j=1
(4.20)
j=1
Proof: If r = 2, this follows from the definition of linearity, or the remark made on homogeneity
following this definition. The general case is proved by induction:
r
r
r
X
X
X
f
xj xj = f x1 x1 + 1
xj xj = x1 f (x1 ) + f
xj x j ,
j=1
j=2
j=2
146
where in the last equality we have used the definition of linearity. Now making the inductive hypothesis that the theorem is true for all linear combinations of r 1 or fewer vectors,
r
r
X
X
f
xj x j =
xj f (xj ) .
j=2
j=2
Combining results, we have proved (4.20).
4.2.1
The matrix representation of linear functions
Let f : Rn Rm be linear. Theorem 38 tells us that for any x =
n
X
xj ej Rn ,
j=1
f (x) = f
n
X
xj e j =
j=1
n
X
xj f (ej ) .
(4.21)
j=1
The right hand side of (4.21) is a a linear combination of the n vectors

f (ej )
j = 1, . . . , n .
All of the data needed to compute f (x) for any x = (x1 , . . . , xn ) Rn is the list of the n vectors
f (ej ) for j = 1, . . . , n.
Example 66 (Evaluating a linear function f given f (ej ), j = 1, . . . , n.). Consider a linear function
f : R3 R3 such that
f (e1 ) = ( 2, 1, 2)
f (e2 ) = (1, 2, 2)
and
f (e3 ) = (1, 1, 1) .
(4.22)
Then,
f (2, 3, 4)
f (2e1 + 3e2 + 4e3 )
2f (e1 ) + 3f (e2 ) + 4f (e3 )
2( 2, 1, 2) + 3(1, 2, 2) + 4(1, 1, 1)
= (3, 0, 14) .
This feature of linear functions is the essence of their simplicity. Though there are infinitely many
possible inputs x at which a linear function f might be evaluated, once you know the values of f (ej )
for each j {1, . . . , n}, you know how to evaluate f (x) for any x Rn .
Definition 51 (Matrix of a linear transformation). Let f : Rn Rm be linear. The matrix Af
corresponding to f is the list of the n vectors {f (e1 ), . . . , f (en )} in Rm , which is written as an m n
array with f (ej ) in the jth column of the array. We express this by writing
Af = [f (e1 ), . . . , f (en )] .
(4.23)
An m n matrix A is any such m n array of numbers. The i, jth entry in the the matrix A is
denoted Ai,j .
147
Example 67. Let f be the function from R3 to R3 given by (4.22). Then placing the three vectors
f (e1 ), f (e2 ) and f (e3 ), respectively, as the columns in a
2
1
Af :=
1
2
2
2
3 3 array, we have
1
.
1
With regard to most problems we shall encounter here, it is more helpful to think of matrices as
lists of vectors rather than rectangular arrays of numbers. That is, we will find it useful to think of
the matrix Af in Example 67 as a list of three vectors:
Af = [v1 , v2 , v3 ] ,
and sometimes to write the vectors themselves as vertical lists of numbers, departing from our practice
up to now of writing vectors as horizontal lists of numbers. In this vertical list notation, we would
write, still referring to Example
v1 = 1
2
67,
v2 =
2
and
v3 =
1 .
1
1
Example 68 (The matrix of a Householder refelction in R3 ). Let u = ( 1, 1, 1), and con3
sider the Householder reflection hu . We have seen in Chapter One that x 7 hu (x) is a linear
transformation from R3 to R3 . What is the 3 3 matrix that represents this linear transformation?
To answer this question, we need only compute hu (e1 ), hu (e2 ) and hu (e3 ), and place these
vectors in the first, second and third columns of a 33 matrix. From the formula hu (x) = x2(du)u,
it is easy to compute hu (e1 ), hu (e2 ) and hu (e3 ), and hence
1 2 2
1
Ahu =
2 1
2
3
2 2
1
the corresponding matrix. One finds:
This is the 33 matrix representing the linear transformation hu . Make sure you do the computations
yourself, in full detail.
Example 69 (The matrix of a cross product transformation). Let b R3 be given by
b = (a, b, c) .
Consider the function f : R3 R3 given by
f (x) = b x .
We have seen in Chapter One that for any s, t R and x, y R3 ,
b (sx + ty) = s(b x) + t(b y) .
148
Thus, f is linear. What is the 3 3 matrix that represents this linear transformation?
To answer this question, we need only compute f (e1 ), f (e2 ) and f (e3 ), and place these vectors
in the first, second and third columns of a 3 3 matrix. The result is:
0 c
b
Af =
0 a
.
c
b
a
0
Make sure you do the computations yourself, in full detail.
Example 70 (The identity matrix). The next example is really simple, but also really important.
The identity transformation Id on Rn is given by
Id(x) = x .
What could be more simple?
Since, by definition,
Id(sx + ty) = sx + ty = sId(x) + tId(y) ,
the identity transformation is linear. What is the corresponding n n matrix?
Since Id(ej ) = ej , it is the n n matrix with ej , written as column vector, in the jth column.
For example, the 4 4 identity matrix, representing the identity transformation on R4 , is
0
Id44 =
0
0
.
0
So far, we have seen that a list of the vectors [f (e1 ), . . . , f (en )] is all that one needs to compute
f (x) given x = (x1 , . . . , xn ). How can we automate this computational process?
The multiplication of matrices and vectors is defined so that if Af is the matrix corresponding to the
linear function f , the matrix product Af x, yields f (x), the value of the function f at x.
Definition 52 (Matrix-vector multiplication). Let A = [v1 , . . . , vn ] be an m n matrix, where the
jth column of A is the vector vj Rm . For every vector x = (x1 , . . . , xn ) Rn , the product of the
matrix A and the vector x is the vector
Ax = [v1 , . . . , vn ](x1 , . . . , xn ) =
n
X
xj vj .
j=1
In particular, for a linear function f : Rn Rm , and Af = [f (e1 ), . . . , f (en )], we have
n
n
X
X
Af x = [f (e1 ), . . . , f (en )](x1 , . . . , xn ) =
xj f (ej ) = f
xj ej = f (x) .
j=1
j=1
(4.24)
149
Thus, the definition of matrix-vector multiplication has been arranged so that

f (x) = Af x .
(4.25)
By Defintion 52, for any m n matrix A, Aej is the jth column of A. Since Ai,j is by definition
the ith entry of the jth column of A, and since the ith entry of any vector y Rm is given by
y1 = ei y, it follows that
Ai,j = (Aej )i = ei Aej .
(4.26)
So far, we have considered m n matrices as lists of n vectors in Rm , through the relation

A = [Ae1 , . . . , Aen ].
It is also profitable to consider an m n matrix as a list of m vectors in Rn , namely the n rows
in A. For example, if
A=
0
.
1 1
the three rows of A, a1 , a2 and a3 are

a1
= ( 2, 1, 1, 5 )
a2
= ( 1, 2, 1, 0 )
a3
= ( 2, 2, 1, 1 ) .
(4.27)
Thinking of A as a list of its rows allows us to think of matrix multiplication in terms of the dot
product, and brings geometry into the picture. The key observation is that the ith entry of Ax is
the dot product of the ith row of A with x. Indeed, the ith entry of Ax is given by
n
n
n
X
X
X
(Ax)i = ei Ax = ei
(Aej )xj =
(ei Aej )xj =
Ai,j xj
j=1
j=1
(4.28)
j=1
Defining the vector

ai = (Ai,1 . . . , Ai,n ) ,
(4.29)
we can rewrite the conclusion of (4.28) as

(Ax)i = ai x .
(4.30)
Along the way to (4.30) we encountered another formula, with which you may be familiar:
(Ax)i =
n
X
Ai,j xj .
(4.31)
j=1
The formulae (4.30) and (4.31) are two ways of expressing the same thing. If one primarily thinks
of m by n matrices as m by n rectangular arrays of numbers, then (4.31) is natural. And it certainly
has its uses.
However, the geometric interpretation of the dot product together with (4.30) will allow us to
use geometric methods to solve equations involving linear transformations. This turns out to be far
more useful that one might expect. Let us summarize:
150
a1
.
.
Theorem 39 (Matrix vector multiplication in terms of matrix rows). Let A =
. be an n m
am
n
matrix expressed as a list of its m rows. Then for any x R ,
Ax = (a1 x, . . . , am x) .
The row representation is closely connected with our practice of writing any sort of function
from Rn to Rm in the form
f (x) = (f1 (x), . . . , fm (x)) ,
that is, in terms of a list of m functions from Rn to R. As you see, each fi (x) is given by
fi (x) = ei f (x) .
(4.32)
n
m
Now suppose that f happens to be the linear transformation from
R to R given by an m n
a1
.
.
matrix A by f (x) = Ax. Let us write A in terms of its rows: A =
. . Then by (4.28), (4.29)
am
and (4.30), (4.33) becomes
fi (x) = ei Ax = ai x .
(4.33)
Now that we know all about the relation between matrices and linear transformations, we can
restate Definition 50, the definition of linear transformations from Rn to Rm as follows:
A function f from Rn to Rm is linear if and only if for some m n matrix A, and all x Rm ,
f (x) = Ax .
As we shall see, there are many convenient matrix manipulation methods for solving equations
such as Ax = b. In fact, linear algebra provides a complete set of methods for answering almost
any question concerning linear transformations. There is no such complete theory for non-linear
transformations, which is why linear approximation is so important.
Thus, linear algebra is an essential part ot of the theory of multivariable calculus, and we shall
introduce many aspects of linear algebra as we develop the theory of multivariable calculus. Of course
in single variable calculus, m = n = 1, and there is not much to say about 1 1 matrices: Linear
algebra in one variable is trivial, and it does not get mentioned by name in single variable calculus.
But already with two variables, it plays an essential role.
Before proceeding, let us be clear on the notation we shall use: Some texts make a distinction
between row vectors and column vectors. We shall not. Instead, we shall simply write vectors in Rn
in either row or column form, using whichever notation seems convenient at the time. In particular,
the dot product of two vectors r and x means exactly what it meant in Chapter 1, even if now we
write one vector as a row and the other as a column.
151
Finally, we can not only multiply matrices, we can add and subtract them and multiply them by
numbers. Indeed, let A and B be two m n matrices, and s and t be two numbers. Let fA and fB
be the linear transformations corresponding to A and B respectively. Then the function sfA + tfB is
the function defined by
(sfA + tfB )(x) = sfA (x) + tfB (x) = sAx = tBx ,
is easily seen to be linear. Thus, it has a matrix representation. It is easy to see that the matrix
representing sfA + tfB has the entries sAi,j + tBi,j . (Simply apply the transformation to ej to find
the jth column.) Therefore:
Definition 53 (Linear combinations of matrices). Let A and B be two m n matrices, and s and t
be two numbers. Then sA + tB is the m n matrix such that for each i, j,
(sA + tB)i,j = sAi,j + tBi,j .
We define A B to be A + (1)B.
The next subsection concerns the solution of the equation f (x) = b when f is a linear function
from Rn to Rn , and provides explicit formulas for n = 2 and n = 3. There is no such general theory
for non-linear functions.
4.2.2
Solving the equation Ax = b for m = n
Let f be a linear function from Rn to Rn . Let A := Af be its n n matrix. Let b be a given vector in
Rn . A basic problem of linear algebra is to find all vectors x Rn , if any, that satisfy the equation
Ax = b ,
or what is the same thing, finding all vectors x Rn , if any, that satisfy f (x) = b.
It is especially important to determine when f is invertible; i.e., when for each b Rn there is
exactly one x Rn such that f (x) = b. The following theorem is fundamentally important in this
regard.
Theorem 40. Let f be a linear function from Rn to Rn . Then if f is one-to-one, it also maps Rn
onto Rn , and hence is invertible.
We will give a proof of a more general result in the final section of this chapter. Geometric proofs
for n = 2 and 3 are sketched in the exercises. These proofs a very illuminating, and so these exercises
should not be skipped.
In the meantime, let us accept the validity of Theorem 40 and draw some consequences from it.
Suppose f is an invertible linear transformation from Rn onto Rn . Then its inverse, f 1 is also linear.
Indeed, for any numbers a, b and anf x, y Rn , f 1 (ax + by) is the unique vecotr z such that
f (z) = ax + by .
(4.34)
152
But since f is linear,

f (af 1 (x) + bf 1 (y)) = af (f 1 (x) + bf (f 1 (y) = ax + by .
Thus, z = af 1 (x) + bf 1 (y) solves (4.34), and hence
f 1 (ax + by) = af 1 (x) + bf 1 (y)
This proves that f 1 is linear. This is worth remembering:
The inverse of an invertible linear transformation is linear.
Now let A := [v1 , . . . , vn ] be the matrix of an invertible linear transformation f from Rn to Rn .
Let B be the n n matrix representation of its inverse. (This makes sense since the inverse is linear,
so it has a matrix). Write B in its row form:
w1
.
.
B=
. .
wn
(4.35)
Since for each j = 1, . . . , n, vj = Aej = f (ej ), it follows that

f 1 (vj ) = Bvj = ej .
By Theorem 39, it follows that
Bvj = (w1 vj , . . . , wn , vj ) = ej .
That is, we must have:
1
wi vj =
0
i=j
(4.36)
i 6= j
This motivates the following theorem:

Theorem 41. Let A := [v1 , . . . , vn ] be the matrix of a linear transformation f from Rn to Rn . Then
f is invertible if and only if there exists a set {w1 , . . . , wn } of n vectors in Rn such that (4.36) is
satisfied. In this case, the matrix of f 1 is given by (4.35), and the set {w1 , . . . , wn } is unique.
Proof: By what we have explained above, when f is invertible, there does exist such a set {w1 , . . . , wn }.
Since {w1 , . . . , wn } gives a formula for the inverse of f , and since there is only one inverse of f , there
is exactly one such set {w1 , . . . , wn }.
Conversely, suppose that such a set {w1 , . . . , wn } exists. Define a function g from Rn to Rn by
g(z) := (w1 z, . . . , wn z) .
Since by definition, f (x) =
n
X
xj vj ,
j=1
g (f (x))
= (w1
n
X
xj vj , . . . , wn
j=1
= (
n
X
xj (w1 vj ), . . . ,
j=1
= (x1 , . . . , xn ) = x ,
n
X
xj v j )
j=1
n
X
xj (wn vj ))
j=1
(4.37)
153
where in the last line we have used (4.36).

This shows that f is one to one, since if f (x) = f (y), then g(f (x)) = g(f (y)), and then by (4.37),
x = y. Then by Theorem 40, f is invertible. Now that we know that f is invertible, (4.37) shows
that g is the inverse of f , and that the matrix representation of g is (4.35).
Before proceeding to the applications, lets us pause for an important point. It may seem that
once we have derived (4.37), we have already proved that g is the inverse of f , and had no need to
invoke Theorem 40. This is not the case.
Indeed, consider the two functions f and g from the set N of the natural numbers into itself that
are given by
f (n) = n + 1
and
g(n) =
n=1
n 1
n>1
(4.38)
Then
g(f (n)) = n for all
nN,
but neither f nor g is invertible. Indeed, there is no n N with f (n) = 1, so that f does not
transform N onto N. Also, g(1) = g(2) = 1, so that g is not a one-to-one transformation of N into
into N, though it does transform N onto N.
The pathology seen in this example does not occur for transformations of finite sets into themselves. Let X denote the finite set X := {1, . . . , N } where N is some positive integer. Let f be
a function defined on X with values in X; i.e., a transformation from X to X. It is not hard to
show that if f is one-to-one, then f is necessarily onto, and that if f is onto, then f is necessarily
one-to-one. This is left as an exercise.
Thus, for a transformation f on a finite set, to check for invertibility, it suffices to check either
whether f is one-to-one, or whether f is onto. For transformations on infinite sets, such as R3 , this
is not necessarily the case, as the example (4.38) shows. However, linear transformations are very
special, and for them we have Theorem 40.
Now let us apply Theorem 41. We begin with n = 2. Let
"
#
a b
A=
so that
A = [v1 , v2 ] ,
c d
where where the columns of A are
v1 = (a, c)
and v2 = (b, d) .
First observe that if v1 = 0, then Ae1 = A0 = 0, so that A is not one-to-one, and therefore not
invertible. Similar considerations apply to v2 , and so if A is invertible, neither column is the zero
vector. Therefore, in trying to find an inversion formula for A, we may suppose neither column of A
is the zero vector.
In order to have w1 v2 = 0 and w2 v1 = 0, we must have that w1 and w2 are multiples of v2
and v1 , respectively:
w1 = ( d, b)
and w2 = ( c, a) .
154
We then compute
w1 v1 = (bc ad)
and w2 v2 = (ad bc) .
Thus, it is possible to achieve (4.36) if and only if ad bc 6= 0, and in this case we have
w1 =
1
(d, b)
ad bc
and w2 =
1
( c, a) .
ad bc
(4.39)
Now applying Theorem 41, we conclude:

Theorem 42. Let f be the linear transformation from R2 to R2 whose matrix is
"
#
a b
A=
.
c d
Then f is invertible if and only if ad bc 6= 0, and in this case the inverse transformation has the
matrix
A
1
:=
ad bc
"
#
.
We now turn to n = 3. Let A = [v1 , v2 , v3 ]. As before, if A is invertible no column is the zero

vector. But even more is true: Suppose v2 is a multiple of v1 ; say v2 = tv1 for some t R. But then
A(e2 te1 ) = Ae2 tAe1 = v2 tv1 = 0 = A0
and since e2 te1 = ( t, 1, 0) 6= (0, 0, 0), the transformation sending x to Ax is not one to one,
and hence is not invertible. It follows that if A is invertible, then no column of A can be a multiple
of any other column of A, or, what is the same thing,
vi vj 6= 0
1i<j3.
for each
(4.40)
Therefore, let us assume (4.40), and try to find an inversion formula for A.
Theorem 41 tells us what to look for: We first seek a vector w1 that is orthogonal to v2 and v3 ,
and also such that w1 v1 = 1. Because of (4.40), to achieve the orthogonality, we must choose w1
to be a multiple of v2 v3 . We can then achieve w1 v1 = 1 if and only if v1 v2 v3 6= 0, in which
case we must choose
1
v2 v3 .
v1 v2 v3
This determines w1 . The same sort of reasoning determines w2 and w3 . At first it might appear
w1 =
that one gets different normalization factor each time, but by the properties of the triple product,
v1 v2 v3 = v2 v3 v1 = v3 v1 v2 ,
and hence one divides by the same quantity in each case. We arrive at:
Theorem 43. Let f be the linear transformation from R3 to R3 whose matrix is A = [v1 , v2 , v3 ].
Then f is invertible if and only if v1 v2 v3 6= 0, and in this case the inverse transformation has
the matrix
A1 :=
v2 v3
1
v3 v1 .
v1 v2 v3
v1 v2
155
The matrix A1 corresponding to the inverse of the transformation x 7 Ax is called the matrix
inverse of A.
The two theorems we have just proved bring us to the following definition:
"
#
a b
Definition 54 (Determinants of 22 and 33 matrices). Let A =
. Then the determinant
c d
of A, det(A), is defined by det(A) := adbc. Let A = [v1 , v2 , v3 ] be a 33 matrix, specified as a list of
its three column vectors in R3 . Then the determinant of A, det(A), is defined by det(A) := v1 v2 v3 .
The nomenclature is justified by the fact that if A is a 2 2 or a 3 3 matrix, then whether the
corresponding linear transformation f (x) := Ax is invertible or not is determined by whether det(A)
differs from zero or not. We shall later define the determinant function on n n matrices for all n
so that the corresponding statement is true.
4.2.3
The distance between two matrices
Approximation and convergence are basic notions in analysis. In this subsection, we introduce a
simple but useful notion of the distance between two m n matrices. This will enable us to give
meaning to the notion of convergence of an infinite sequence of matrices, which will be useful for
solving many problems. We begin with a deifnition of a quantity that measures the size of a matrix
in a natural way.
Definition 55 (Frobenius norm of a matrix and Frobenius distance). Let A be an m n matrix.
The non-negative number kAkF defined by
1/2
n X
m
X
kAkF =
|Ai,j |2
j=1 i=1
is called the Frobenius norm of the matrix A.

The Frobenius distance between two m n matrices A and B is the quantity dF (A, B) given by
dF (A, B) = kA BkF .
Note that if we may identify an m n matrix with a vector in Rmn by the "simple device
# of
1 2 3
writing out the rows, one after another in a long vector vA . For example, for A =
, we
4 5 6
would have
vA = (1, 2, 3, 4, 5, 6) .
Notice that the definition is set up so that kAkF = kvA k. That is, the Frobenius norm of A is nothing
other than the length of the vector one gets by stretching A out as a vector vA Rnm
Notice also that because of the way matrix addition (and subtraction) is defined, for any two
m n matrices A and B,
vAB = vA vB .
156
It follows from this that the Frobenius distance dF satisfies the triangle inequality
dF (A, C) dF (A, B) + dF (B, C)
for all m n matrics A, B and C
Since it is clear that dF (A, B) = 0 if and only if A = B, and that dF (A, B) = dF (B, A), this
shown that dF (A, B) is a metric on the set of m n matrices. It is therefore correct to refer to it as
a distance.
The set of m n matrices equipped with the Frobenius distance dF therefore provides us with an
example of a metric space albeit one that is very closely connected with our only other example at
present, Euclidean space. Now we can not only do algebra with matrices: We can do analysis, and
use limiting processes to solve problems.
The Frobenius distance is useful to us because of its very close connection with the Euclidean
distance, and because of the following theorem which says that it behaves well with respect to matrix
multiplication:
Theorem 44 (The Frobenius norm and matrix multiplication). Let A be an m n matrix, and let
B be an n p matrix so that the matrix product AB is defined. Then
kABkF kAkF kBkF .
(4.41)
Proof: By the Cauchy-Schwarz inequality,

n
X

|(AB)i,j | =
Ai,k Bk,j

n
X
k=1
!1/2
A2i,k
k=1
n
X
!1/2
2
Bk,j
k=1
Therefore,
kABk2F
p
m X
X
(AB)2i,j
i=1 j=1
m
n
X
X
i=1
!
A2i,k
p
n
X
X
j=1
k=1
!
2
Bk,j
= kAk2F kBk2F .
k=1
Let C be an n n matrix, so that we can form powers of C. Let us define C 0 to be the identity
matrix. Consider the sum
N
X
Ck .
k=0
The same telescoping sum trick for evaluating this sum when C is a number also works when C is
a matrix: Multiply by (I C) and distribute:
!
N
N
N
+1
X
X
X
k
(I C)
C
=
Ck
C k = I C N +1 .
k=0
k=1
k=1
That is,
(I C)
N
X
k=0
!
C
I = C N +1 .
(4.42)
157
Now suppose that kCkF < 1. Then by Theorem 44,

!
N

X

+1
,
C k I = kC N +1 kF kCkN
(I C)
F

k=0
(4.43)
and the right hand side goes to zero as N goes to infinity. This suggests that I C is invertible, and
that
N
X
lim
!
C
= (I C)1 .
k=0
This is the case.

Theorem 45 (Geometric series for matrices).
be
Let C !
N
X
each 1 i, j n the sequence of numbers
Ck
k=0
matrix
an
n n matrix with kCkF < 1. Then for
is convergent, so that we may define a
i,j
C k by
k=0
!
C
k=0
N
X
= lim
i,j
The matrix (I C) is invertible, and
!
C
k=0
(C k )i,j .
(4.44)
k=0
i,j
C k is its inverse, and
k=0
1
.
(4.45)
1 kCkF
!
N
X
Proof: Since the real numbers are complete, to show that

Ck
is convergent, is suffices
k(I C)1 kF
k=0
i,j
to show that this sequence is Cauchy. That, for each > 0, we must show that there exists an N so
that
M, N > N

!
X
N k

C

k=0
i,j
!

<.
Ck

k=0
i,j
M
X
(4.46)
Note that for M < N ,

N
X
k=0
!
Ck
i,j
M
X
!
Ck
k=0
=
i,j
N
X
Ck
k=M +1
i,j

N
X

Ck

k=M +1
where we have use the fact that for any matrix A and any i, j, |Ai,j | kAkF , which is an obvious
consequence of the definition. But by the triangle inequality for the Frobenius norm (the Euclidean
triangle inequality in disguise), and then Theorem 44,
N

N
X

X
k

k
C
C

F

k=M +1
Recall that for any number 0 < c < 1,
k=M +1
N
X
k=M +1
N
X
kCkF .
k=M +1
X
k=M +1
ck =
cM +1
. Hence, choosing N to be
1c
the least integer such that (cN )/(1 c) < , we have that (4.46) is satisfied for this choice of N .
Hence, the sequence is convergent.
158

Next, from (4.43) and once more the fact that for any matrix A and any i, j, |Ai,j | kAkF ,
!
!!
!!
N
N
X
X
X
k
k
k
(I C)
C
= lim (I C)
C
= lim (I C)
C
= Ii,j ,
k=0
k=1
i,j
and hence
(I C)
k=1
i,j
i,j
!
C
=I .
k=0
This shows that for any x Rn , if y :=
C k x, then (I C)y = x. Thus, I C is onto, and hence
k=0
invertible by Theorem 40. By what we have just seen,
C k is the inverse.
k=0
Finally, by the triangle inequality for the Frobenius norm , and then Theorem 44 once more,
k
C k kF
k=0
kCkkF =
k=0
1
.
1 kCkF
Theorem 45 provides a very useful formula for matrix inverses for arbitrarily large square matrices. Here is one important application, which says that if A is invertible, all matrices B that are
sufficiently close to A, as measured by the Frobenius distance, are invertible.
Theorem 46 (Continuity of invertibility). Let A be an invertible n n matrix. Then every n n
matrix B such that
kB AkF < kA1 k1
F
is invertible with
kB

kF
(4.47)
kA1 k2F
1 kA1 kF kA BkF

kA BkF .
(4.48)
Proof: Define C := A1 (A B). Then

B = A + (B A) = A(I C) .
By Theorem 44, kCkF kA1 kF kA BkF , so that when (4.47) is true, then kCkF < 1. Thus by
Theorem 45, C is invertible, Since the product of invertible matrices is invertible, B = A(I C) is
invertible, and the inverse is B 1 = (I C)1 A1 . By Theorems 44 and 45 once more,
kB 1 kF k(I C)1 kF kA1 kF
kA1 kF
.
F kA BkF
kA1 k
(4.49)
Finally, applying Theorem 44 to the identity

B 1 A1 = B 1 (B A)A1
and then using (4.49) to bound kB 1 kF , we obtain (4.48).
Theorem 46 says that the set of invertible n n matrices is an open subset in the metric space
that is set of all n n matrices equpped with the Frobenius distance: For any invertible A, the
ball consisting of all n n matrices B with dF (A, B) < kA1 k1
F is included in the set of invertible
4.3. DIFFERENTIABILITY OF FUNCTIONS FROM RN TO RM
159
matrices. Hence this set is open. Theorem 46 also says that the function A 7 A1 taking the
invertible n n matrices into itself is continuous: If A is invertible and > 0, then (4.48) provides a
such that
kB AkF <
kB 1 A1 kF < .
We close this subsection with one final theorem on the continuity of matrix multiplication. It
can be seen as a special case of Theorem 44 if we think of vectors in Rn as n 1 matrices, but we
give an independent proof anyway.
Theorem 47. Let A be an m n matrix. Then for all x, y Rn ,
kAx Ayk kAkF kx yk .
(4.50)
In particular, every linear transformation is Lipschitz continuous, and the Lipschitz constant is no
larger than the Frobenius norm of the corresponding matrix.
Proof: Let a1 , . . . , am denote the m rows of A. Then,
Ax Ay = A(x y) = (a1 (x y), . . . , am (x y)) .
Therefore, kAx Ayk2 =
m
X
(ai (x y)) . By the Cauchy-Scwarz inequality, |ai (x y)|
i=1
kai kkx yk. Therefore,

2
kAx Ayk
m
X
!
2
kai k
kx yk2 .
i=1
Now using the fact that for each i, kai k2 =
n
X
|Ai,j |2 , we obtain (4.50).
j=1
In particular, if f is a linear transformation from Rn to Rm , and Af is its corresponding matrix,

then for any > 0,
kx yk <

kAf kF
kf (x) f (y)k < .
While finding explicit values of to go with given values of can be a chore for non-linear functions,
it is automatic in the linear case.
We have now learned enough about linear functions that we are ready to return to differentiation.
4.3
4.3.1
Differentiability of functions from Rn to Rm

Differentiability and best linear approximation in several variables
Definition 56 (Differentiable functions from Rn to Rm ). A function f from Rn to Rm is differentiable

at x0 in case there is some mn matrix A from Rn to Rm such that for each > 0, there is a () > 0
so that
kx x0 k < ()
kf (x) f (x0 ) A(x x0 )k < kx x0 k .
(4.51)
160

In other words, f is differentiable at x0 when for all x sufficiently close to x0 , the deviation of
f from its value at x0 is given by

A(x x0 )
for some n n matrix A, up to errors that are negligibly small compared to kx x0 k, where we have
chosen to be negligible. Thus, we have the linear approximation
f (x) f (x0 ) + A(x x0 ) .
(4.52)
This is a very important definition. It may look different from what you may have expected. We
shall now very carefully examine it and become completely familiar with it.
First of all, suppose A and B are two m n matrices and for all > 0, there is a () > 0 so
that whenever kx x0 k < (), both
kf (x) f (x0 ) A(x x0 )k < kx x0 k
and
kf (x) f (x0 ) B(x x0 )k < kx x0 k
are true, Then, for such x, we have from the triangle inequality,
k(A B)(x x0 )k
= k[f (x) f (x0 ) B(x x0 )] [f (x) f (x0 ) A(x x0 )]k
kf (x) f (x0 ) B(x x0 )k + kf (x) f (x0 ) A(x x0 )k
2kx x0 k .
Next, for any j, if 0 < t < (), then x := x0 + tej satisfies kx x0 k = t < (). Evaluating both
sides above for this choice of x, and cancelling a factor of t, we obtain k(A B)ej k . Since can
be chosen arbitrarily small, we conclude k(A B)ej k = 0 for each j. But this means that Aej = Bej
for each j, and hence A = B. In summary:
There can be at most one n m matrix A for which (4.51) is true for all > 0.
The uniqueness of A that we have just proved justifies referring to the approximation in (4.51) as
the best linear approximation to f at x0 : It is the only one for which (4.51) is true. Any other linear
approximation entails errors that are not negligible at the first order in kx x0 k. The uniqueness
also allows us to make the following definition.
Definition 57 (Derivative and continuity of derivatives). Let f be a function from Rn to Rm that is
differentiable at some x0 Rn . Then the unique m n matrix A such that 4.51 is true for all > 0
is called the derivative of f at x0 . It is denoted by [Df (x0 )].
Furthermore, if f is differentiable in an open set U , then f is continuously differentiable in U in
case the function x 7 [Df (x)] is continuous form U to the space of m n matrices in the sense that
x 7 [Df (x)]i,j is continuous form U to R for each i, j.
So far we know that the derivative [Df (x0 )] is unique when it exits. But how do we decide when
it exists, and how do we compute it? Also, how is this notion of derivative related to the notions of
directional derivatives and partial derivatives? To answer these questions, let us look again at the
definition.
161
Let us pick some t 6= 0, some 1 j n, and set

x := x0 + tej .
With this choice of x,
kx x0 k < ()
kf (x) f (x0 ) A(x x0 )k < kx x0 k
becomes
|t| < ()
kf (x0 + tej ) f (x0 ) tAej k < |t| ,
which is equivalent to
|t| < ()

f (x0 + tej ) f (x0 )

Ae
j < .

t
This is exactly the , definition of

lim
t0
f (x0 + tej ) f (x0 )

= Aej .
t
(4.53)
Therefore, if f is differentiable at x0 , (4.53) gives us a formula for the jth column of A = [Df (x0 )].
Moreover, this formula is clearly related to partial differentiation. To bring this out, we look at (4.53)
component by component. Let us write
f (x) = (f1 (x), . . . , fm (x))
so that fi (x) = ej f (x) for each i = 1, . . . , n. Taking the dot product of both sides of (4.53) by ei
we obtain
Ai,j = ei Aej = lim
t0
fi (x0 + tej ) fi (x0 )
=
fi (x0 ) .
t
xj
We conclude that if f is differentiable at x0 , then each of the partial derivatives
fi (x0 )
xj
1im
, 1jn
exist, and for each such i and j,

[Df (x0 )]i,j =
fi (x0 ) .
xj
There is a nice way to write this: Notice that the ith row of [Df (x0 )] is fi (x0 ), so that
f1 (x0 )
..
.
[Df (x0 )] =
.
fm (x0 )
(4.54)
One might hope that if all of the partial derivatives of f exist at x0 , then f would be differentiable
at x0 . However, this is not the case. Fortunately, it is the case if one assumes only a little more:
Theorem 48 (Differentiability of f and partial derivatives). Let f = (f1 , . . . , fm ) be a function from
Rn to Rm . Suppose that on some open set U including x0 , all of the partial derivaitves of each
component fi of f exist, and are continuous at x0 . Then f is differentiable at x0 .
162

Before giving the proof, we give an application:
Example 71 (Differentiability of rational functions). Let p(x) and q(x) be polynomials in the n
variables x1 , . . . , xn . Let U be an open set in Rn such that q(x) 6= 0 for any x U . Then the rational
function
f (x) :=
p(x)
q(x)
is well-defined on U .
By the quotient rule of single variable calculus, for eacj j = 1, . . . , n,
f (x) =
xj
q(x)
xj
2

p(x) q(x)
q(x) p(x) .
xj
xj
p(x) and
q(x) are both polynomials in the n variables x1 , . . . , xn , so is
f (x),
xj
xj
xj
and the denominator is not zero for any x U . Therefore, each of the partial derivatives of f is
Since
continuous in U , and hence f is differentiable at each point x in U .

As a special case, f (x, y) = x2 + y 2 is differentiable at each (x0 , y0 ) in R2 .
f1 (x0 )
..
. We
Proof of Theorem 48: Pick > 0. Let A be the m n matrix given by A :=
.
fm (x0 )
must find a () > 0 so that (4.51) is true.
Let us divide and conquer. First note that
kf (x) f (x0 ) A(x x0 )k
m
X
|fi (x) fi (x0 ) fi (x0 ) (x x0 )| .
i=1
Therefore, if for each i we can find a i () > 0 so that

kx x0 k < i ()
|fi (x) fi (x0 ) fi (x0 ) (x x0 )| <

kx x0 k ,
m
then definiing
() = min{i (), . . . , m ()} ,
we shall have established (4.51).
This means that it suffices to prove the m = 1 case of the theorem. We now do this, in
close analogy with the proof of Theorem 36, though one can also extend the method used to prove
Theorem 37 to more variables. As before, we give the proof for n = 2 to avoid complicated notation.
Once this is understood, it will be clear how the extension to n > 2 goes.
To begin, fix x and x0 and define numbers a and b by
x x0 = a
and
y y0 = b
Define the function h(x) by

h(x) = f (x0 ) + (x x0 ) f (x0 ) .
163
We must show that for this choice of h,

lim
xx0
kh(x) f (x)k
=0.
kx x0 k
(4.55)
Define single variable functions and by

(s) = h(x0 , y0 + sb) f (x0 , y0 + sb)
and
(t) = h(x0 + ta, y0 + b) f (x0 + ta, y0 + b) .
As you can check,

[(1) (0)] + [(1) (0)]

f (x, y) h(x, y) h(x0 , y) f (x0 , y)

+
f (x0 , y) h(x0 , y) f (x0 , y0 ) h(x0 , y0 )
=
[h(x) f (x)] [h(x0 ) f (x0 )] .
=
Since h(x0 ) = f (x0 ),
h(x) f (x) = [(1) (0)] + [(1) (0)] .
(4.56)
Now apply the Mean Value Theorem to the single variable functions (t) and (s) to find
[(1) (0)] = 0 (t)
and
[(1) (0)] = 0 (
s)
(4.57)
for some s and t between 0 and 1.

Using the chain rule, and our explicit formula for h, we compute

f
h
(x0 + t
a, y0 + b)
(x0 + ta, y0 + b) a
0 (t) =
x
x

f
f
=
(x0 , y0 )
(x0 + ta, y0 + b) a .
x
x
Likewise,

f
h
(
(x0 , y0 + sb)
(x0 , y0 + sb) b
s) =
y
y

f
f
(x0 , y0 )
(x0 , y0 + sb) b
=
y
y
0
Combining (4.56) and (4.57),

f
h(x) f (x) =
(x0 , y0 )
x

f
+
(x0 , y0 )
y

f
(x0 + ta, y0 + b) a
x

f
(x0 , y0 + sb) b .
y
The right hand side is the dot product of

f
f
f
f
(x0 , y0 )
(x0 + ta, y0 + b) ,
(x0 , y0 )
(x0 , y0 + sb)
x
x
y
y
with (a, b) = x x0 .
164
Therefore, by the Schwarz inequality,

|h(x) f (x)|
kx x0 k
s
f (x0 , y0 )
f (x0 + ta, y0 + b)
x
x
2

+
f (x0 , y0 )
f (x0 , y0 + sb)
y
y
2
.
Now, as x x0 , a 0 and b 0. Therefore, using the continuity of the partial derivatives at

this point,
lim
xx0
|h(x) f (x)|
=0,
kx x0 k
as was to be shown.
4.3.2
The general chain rule
Let f be a a function from Rn to Rm that is differentiable at x0 Rn . Let g be a differentiable

function from Rm to R` that is differentiable at f (x0 ). Then for x near x0 ,
f (x) f (x0 ) + [Df (x0 )](x x0 ) ,
and for y near f (x0 ),
g(y) g(f (x0 )) + [Dg (f (x0 )](y f (x0 )) .
If we take y := f (x), then for x sufficiently close to x0 , y will be close to f (x0 ), and so we will have
g(f (x))
g(f (x0 )) + [Dg (f (x0 )](f (x) f (x0 ))

g(f (x0 )) + [Dg (f (x0 )][Df (x0 )](x x0 ) .
Thus, the composite function g f will have the linear approximation

g(f (x)) g(f (x0 )) + [Dg (f (x0 )][Df (x0 )](x x0 )
at x0 , which suggests that gf is differentiable, and its derivative is the matrix product [Dg (f (x0 )][Df (x0 )].
Remembering that matrix multiplication represents composition of linear functions, this formula
would be very natural. In fact, it is valid without any extra hypotheses.
Theorem 49. Let f be a a function from Rn to Rm that is differentiable at x0 Rn . Let g be a
differentiable function from Rm to R` that is differentiable at f (x0 ). The the composite function g f
is differentiable at x0 and
[Dgf (x0 )] = [Dg (f (x0 )][Df (x0 )] ,
where the right hand side is the product of the Jacobian matrices of f and g at the indicated points.
Proof: Fix any > 0. We must show that there is a > 0 such that for v Rn ,
kx x0 k <
kg(f (x)) g(f (x0 )) [Dg (f (x0 )][Df (x0 )](x x0 )k < kx x0 k .
165
Define y := f (x) and y0 := f (x0 ) and then

w

:= f (x) f (x0 ) [Df (x0 )](x x0 )

z := g(y) g(y0 )) + [Dg (y0 ))](y y0 ) .
Then
g(f (x)) g(f (x0 ))
= g(y) g(y0 )
=
[Dg (y0 )](y y0 ) + z
[Dg (y0 )](f (x) f (x0 )) + z

[Dg (y0 ))] [Df (x0 )](x x0 ) + w + z
[Dg (f (x0 ))][Df (x0 )](x x0 ) + [Dg (f (x0 )]w + z .
Thus by the triangle inequality, and then Theorem 44,

kg(f (x)) g(f (x0 )) [Dg (f (x0 ))][Df (x0 )](x x0 )k
k[Dg (f (x0 ))]wk + kzk
k[Dg (f (x0 ))]kF kwk + kzk
(4.58)
We have reduced the problem to showing that for some > 0,

kx x0 k <
k[Dg (f (x0 ))]wk + kzk < kx x0 k .
(4.59)
Now fix any e

> 0. Since f is differentiable at x0 , and g is differentiable at y0 , there is a ee > 0
such that
kx x0 k < ee
kwk < e
kx x0 k ,
(4.60)
ky y0 k < ee
kzk < e
ky y0 k .
(4.61)
and
Next, by the definitions above, y y0 = w + [Df (x0 )](x x0 ), and so

ky y0 k kwk + k[Df (x0 )]kF kx x0 k (e
+ k[Df (x0 )]kF )kx x0 k .
Thus if we define
ee
(e
) := min ee ,
e
+ k[Df (x0 )]kF
)
,
it follows from (4.62) that

kx x0 k < (e
)
ky y0 k < e
and
kx x0 k < e .
Combining (4.3.2), (4.60) and (4.61), we see that

kx x0 k < be
kg(f (x)) g(f (x0 )) [Dg (f (x0 ))][Df (x0 )](x x0 )k

e
k[Dg (f (x0 ))]kF + (e
+ k[Df (x0 )]kF kx x0 k
Now choose e
> 0 so that

e
k[Dg (f (x0 ))]kF + (e
+ k[Df (x0 )]kF < .
Then, with this choice of e
, (4.59) is valid if we use (e
) in place of .
(4.62)
166
4.4
4.4.1
Newtons Method
Linear approximation and Newtons iterative scheme
Let f be a differentiable function from Rn to Rn . Suppose we want to solve the equation f (x) = 0.
For example, if we are given a differentiable function f from Rn to R, and we want to find its critical
points, than we must solve the equation
f (x) = 0 .
Defining f (x) := f (x), we have arrived at the equation f (x) = 0.
The equation f (x) = 0 arises in many other ways as well. Let g and h be functions from Rn to
n
R , and suppose that we want to solve g(x) = h(x). Defining f by f (x) := g(x) h(x), we see that
g(x) = h(x)
f (x) = 0 .
This example is worth bearing in mind: Many equations can be written in the form f (x) = 0 by the
simple device of moving everything to the left of the equal sign.
Newtons method for solving f (x) = 0 is to successively improve an approximate solution using
the linear approximation to f that is provided by differentiation. The multivariable version works in
exactly the same way as the single variable version.
Recall that the linear approximation to f at x0 is given by
f (x) f (x0 ) + [Df (x0 )] (x x0 ) .
(4.63)
Now replace f in the equation f (x) = 0 by its linear approximation to obtain an equation that
approximates (4.63):
f (x0 ) + [Df (x0 )] (x x0 ) = 0 ,
(4.64)
which is the same as

[Df (x0 )] x = [Df (x0 )] x0 f (x0 ) .
This is nothing other than the matrix equation
Ax = b
where
A := [Df (x0 )]
and
v := [Df (x0 )] x0 f (x0 ) .
As long as the matrix A := [Df (x0 )] is invertible, there is a unique solution which is x = A1 b,
or, more explicitly,
1
x = x0 [Df (x0 )]
f (x0 ) .
Now, in so far as x0 is an approximate solution to f (x) = 0, so that we may expect there to be

an exact solution nearby, and in so far as (4.63) is a good approximation, we can expect the solution
of (4.64) to be a more accurate approximate solution. That is, defining
1
x1 := x0 [Df (x0 )]
f (x0 ) ,
we can hope that x1 is a better approximate solution than x0 .
4.4. NEWTONS METHOD
167
Now simply iterate this improvement procedure: Given the starting point x0 , we define the
infinite sequence {xn } by
1
xn+1 := xn [Df (xn )]
f (xn ) .
(4.65)
Of course, the construction of the sequence is only meaningful if [Df (xn )] is invertible for each n; i.e.,
if det([Df (xn )]) 6= 0 for each n.
Newtons method is a successive approximations method. It takes a starting guess for the
solution x0 , and iteratively improves the guess. The iteration scheme produces an infinite sequence
of approximate solutions {xn }. Under favorable circumstances, this sequence will converge very
rapidly toward an exact solution. In fact, the number of correct digits in each entry of xn will more
or less double double at each step, once you get reasonably close. If you have one digit right at the
outset, you may expect about a million correct digits after 20 iterations more than you are ever
likely to want to keep!
To explain the use of Newtons method, we have to cover three points:
(i) How one picks the starting guess x0 .
(ii) How the iterative loop runs; i.e., the rule for determining xn+1 given xn , which we have already
explained.
(iii) How to break out of the iterative loop we need a stopping rule which ensures us that there
is an actual solutions within some prescribed distance of xn , and hence we may stop iterating at
the nth step.
We begin by explaining (ii), the nature of the loop. Once we are familiar with the loop, we can
better understand what we have to do to start it and stop it. Let us run through an example.
Consider the following system of non linear equations:
x2 + 2yx =
xy
4
1.
(4.66)
As noted above, any system of two equations in two variables can be written in the form
f (x, y)
g(x, y)
0.
In this case we define f (x, y) = x2 + 2xy 4 and g(x, y) = xy 1.

Next, introducing f (x) = (f (x), g(x)), we can write (4.66) as a single vector equation f (x) = 0.
In the case of (4.66), we have
f (x, y) = (x2 + 2yx 4 , xy 1) .
(4.67)
In this example, we can solve f (x) = 0 by algebra alone. For this purpose, the original formulation
is most convenient. Using the second equation in (4.66) to eliminate y, the first equation becomes
x2 = 2. Hence x = 2. The second equation says that y = 1/x and so we have two solutions
( 2, 1/ 2)
and
( 2, 1/ 2) .
168
In other examples, it may be quite hard to eliminate either variable, and algebra alone cannot deliver
solutions.
Now let us see how Newtons Method works with this example:
Example 72 (Using Newtons iteration). Consider the system of equations f (x) = 0 where f is
given by (4.67). We will choose a starting point so that at least one of the equations in the system is
satisfied, and the other is not too far off. This seems reasonable enough. Notice that with x = y = 1,
xy 1 = 0, while x2 2xy 4 = 1. Hence with x0 = (1, 1) we have
f (x0 ) = ( 1, 0) .
Now let us write our system in the form f (x, y) = 0. Computing the Jacobian of f , we find that
"
[Df (x)] =
2x + 2y
2x
#
(4.68)
and hence
"
[Df (x0 )] =
1
Hence x1 := x0 [Df (x0 )]
x1 = (1, 1)
#1
"
#
.
(4.69)
f (x0 ) is
"
"
#1
( 1, 0) .
, we find x1 = (3/2, 1/2). Notice that x1 is indeed considerably

= 21
1
4
1 1
closer to the exact solution ( 2, 1/ 2) than x0 . Moreover,
Since
1
f (x1 ) = (1, 1) .
4
This is a better approximate solution; it is much closer to the actual solution. If you now iterate this
further, you will find a sequence of approximate solutions converging to the exact solution ( 2, 1/ 2).
You should compute x2 and x3 and observe the speed of convergence.
4.4.2
The Geometry of Newtons Method
To understand when and how Newtons method works, it is useful to relate the affine approximation
of f to the tangent plane approximation of each of the entry functions fj in f = (f1 , . . . , fn ).
To be able to draw graphs, and to keep the discussion as geometrically clear as possible, let us
take n = 2, and write f = (f, g). Then f = 0 is equivalent to the system of equations
f (x, y)
g(x, y)
0.
(4.70)
4.4. NEWTONS METHOD
169
Replace this by the equivalent system

z
f (x, y)
g(x, y)
0.
(4.71)
From an algebraic standpoint, we have taken a step backwards we have gone from two equations
in two variables to three equations in three variables. However, (4.71) has an interesting geometric
meaning: The graph of z = f (x, y) is a surface in R3 , as is the graph of z = g(x, y). The graph
of z = 0 is just the x, y plane a third surface. Hence the solution set of (4.71) is given by the
intersection of 3 surfaces.
For example, in the plot below you see the three surfaces in (4.71) when f (x, y) = x2 + 2xy 4
and g(x, y) = xy 1, as in Example 72. Here, we have plotted 1.3 x 1.8 and 0.5 y 1, which
includes one exact solution of the system (4.70) in this case. The plane z = 0 is the surface in solid
color, z = f (x, y) shows the contour lines, and z = g(x, y) is the surface showing a grid. You see
where all three surfaces intersect, and that is the where the solution lies.
You also see in this graph that the tangent plane approximation is pretty good in this region,
so replacing the surfaces by their tangent planes will not wreak havoc on the graph. So here is what
we do: Take any point (x0 , y0 ) so that the three surface intersect near (x0 , y0 , 0). Then replace the
surfaces z = f (x, y) and z = g(x, y) by their tangent planes at (x0 , y0 ), and compute the intersection
of the tangent planes with the plane z = 0. This is a simple linear algebra problem that we know haw
to solve. Replacing z = f (x, y) and z = g(x, y) by the equations of their tangent planes at (x0 , y0 )
amounts to the replacement
z = f (x, y)
z = f (x0 ) + f (x0 ) (x x0 )
z = g(x, y)
z = g(x0 ) + g(x0 ) (x x0 )
and
170
where x0 = (x0 , y0 ). This transforms (4.71) into

z
f (x0 ) + f (x0 ) (x x0 )
g(x0 ) + g(x0 ) (x x0 )
0.
(4.72)
Now we can eliminate z, and pass to the simplified system
"
Since [Df (x0 )] =
f (x0 )
f (x0 ) + f (x0 ) (x x0 )
g(x0 ) + g(x0 ) (x x0 )
0.
(4.73)
, this is equivalent to (4.64) by the rules for matrix multiplication.

g(x0 )
We see from this analysis that how close we come to an exact solution in one step of Newtons
method depends on how good the tangent plane approximation is at the current approximate solution.
To really understand how well Newtons method works, we need to understand something about how
the surfaces given by z = f (x, y) and z = g(x, y) curve away from their tangent planes at x0 .
That is, we will need to know something about the curvature of surfaces. As you may guess from
our investigation of curvature of curves x(t), this will involve second derivatives of functions from
Rn to R. We take up the topic of higher derivatives of functions from Rn to R in the next chapter.
However, there is much that can be accomplished considering only first derivatives.
4.4.3
Starting and stopping the iteration
It is not always a simple matter to determine how many solutions f (x) = 0 may have, or to find
approximate locations of these solutions. Ideally, before starting the iterative procedure, one would
like to how many solutions f (x) = 0 has, at least in some region of interest, and then one needs to
find a good starting point close to each of these. Let us begin with a simple example where this is
readily accomplished.
Example 73 (Graphical location of starting points). Consider the system
(f (x, y), g(x, y)) = 0
where f (x, y) = x4 + xy, and g(x, y) = 1 y 2 x2 .
The equation g(x, y) = 0 is equivalent to the equation x2 + y 2 = 1, i.e., the equation of the
unit circle. Therefore, all solutions of g(x, y) = 0 lie on the unit circle, and hence all solutions of
(f (x, y), g(x, y)) = 0 lie on the unit circle.
Now, what about the equation f (x, y) = 0? Can we find a similar explicit description of its
solution set? Yes, since f (x, y) = x(x3 + y), f (x, y) = 0 if and only if x = 0, which is the equation
of the y axis, or if y = x3 , which is the equation of an easily plotted cubic curve.
Hence the solutions of our system of equations are exactly the intersection of the unit circle with
the y-axis, and the intersection of the unit circle with the cubic curve y = x3 . Here is a plot showing
the unit circle, the y-axis and the cubic curve y = x3 .
4.4. NEWTONS METHOD
171
As you see, the system has four solutions. Two of them are readily given in closed form: The
y-axis intersects the unit circle at the points (0, 1) and (0, 1).
It is actually possible to find an exact formula for the other two solutions by algebraic means.
Substituting y = x3 into x2 + y 2 = 1, we obtain x2 + y 6 = 1. Introducing the variable t = x2 , this
becomes the cubic equation t + t3 = 1. There is an algebraic formual, known as Cardanos formula
for the roots of any cubic.
Using Cardanos formula, one would find that the unique real solution of t + t3 = 1 is
t=
1
2
(108 + 12 93)1/3
.
6
(108 + 12 93)1/3
The remaining two solutions have x and y coordinates
x= t=
1
2
(108 + 12 93)1/3
6
(108 + 12 93)1/3
1/2
and then y = x3 .
Numerically evaluating these expressions, one finds the two solutions are given, to the first 10
digits, by
( 0.8260313578 , 0.5636241625)
and
(0.8260313578 , 0.5636241625) .
In many applications, what one would want anyhow would be some such decimal approximation,
and not the exact solution obtained by using Cardanos formula (which is considerably more complicated than the quadratic formula). Newtons method provides a more direct route. From the plot you
can see that
( 0.80 , 0.55)
and
(0.80 , 0.55)
are decent aprroximate solutions. Using either one as the starting point for the Newton iteration,
one readily determines 10 (or more) accurate digits. Moreover, since the second solution is exactly
minus the first, one only need to run the iteration for one of these solutions.
172

If you have more than two variables, plots become harder to use. In such a case, it is often easier
to eliminate from consideration a set of points that cannot possibly contain a solutions, and then to
choose starting points from among the points that are left over. The following lemma tells us where
not to look for solutions.
The basic idea is very simple. Suppose at some point x0 , the ith component of f is not zero; i.e.,
fi (x0 ) 6= 0 .
Suppose for the sake of argument that fi (x0 ) > 0, so that your altitude is too high at x0 . To get to
a solution, you have to get downhill by a height h = fi (x0 ). But if the slope is bounded in magnitude
by C > 0, you cannot lose altitude h without traveling a horizontal distance of at least h/C. Since
any solution x of f (x) = 0 must satisfy fi (x) = 0, there can be no solution within a radius h/C of
x0 . Here is the precise version:
Lemma 11 (Elimination lemma). Let B be a bounded closed subset of Rn . Suppose also that C
is convex; i.e., C contains every point on the line segment connecting any two points in C. Let
f = (f1 , . . . , fn ) be a function from Rn to Rn that is continuously differentiable on some open set
containing B. Let C1 , . . . , Cn be any n positive numbers such that for all x B.
kfj (x)k Cj
for
j = 1, . . . , N ,
(4.74)
Then for any r > 0, and any x0 B, in order that there is an x B with
f (x) = 0
and
kx x0 k < r ,
it is necessary that

max
Proof: Let j be such that r(x0 ) =
fj (x0 )
, j = 1, . . . , n
Cj

r .
fj (x0 )
. For any x B define g(t) by
Cj
g(t) = fj ((1 t)x0 + tx) .
Then g is continuously differentiable on the unit interval, and by the Mean Value Theorem,
g(1) = g(0) + g 0 (c) = f (x0 )
for some c in the unit interval.
By the chain rule, g 0 (c) = fj [(1 c)x0 + cx] (x x0 ), and then by the Schwarz inequality, and
the fact that (1 c)x0 + cx B since B is convex,
|g 0 (c)| kfj ((1 c)x0 + cx)kkx x0 k Cj kx x0 k .
Therefore
|fj (x)| = |g(1)| = |g(0) g 0 (c)|
|g(0)| |g 0 (c)| |fj (x0 )| Cj kx x0 k .
(4.75)
4.4. NEWTONS METHOD
173
We see that kx x0 k < |fj (x0 )|/Cj implies that |fj (x)| 6= 0, and hence that f (x) 6= 0. Hence there
can be no x B with f (x) = 0 and kx x0 k |fj (x0 )|/Cj .
Here is how the lemma can be applied. Suppose f = (f1 , . . . , fn ), and we are looking for solutions
of f (x) = 0 in the set B given by
L xi L for i = 1, . . . , N .
(4.76)
Suppose also that you can find positive numbers C1 , . . . , Cn such that for x satisfying (4.76). Let
C := max{Cj : j = 1, . . . , n } .
Pick any positive number r so that Cr is small . Then since B is closed and convex, for any x0 B,
we have from Lemma 11 that either:
(1): |fj (x0 )| Cr for each j, in which case x0 makes a good choice (since Cr is small) as a starting
point for Newtons method,
or else
(2): kf (x0 )k > 0 for all x B with kx x0 k, in which case no point in the entire ball of radius r
around x0 is a particularly good choice as a starting point for Newtons method, ,
We are ready to state a procedure: First determine a value for C. Often the Cauchy-Scwharz
inequality provides one, as in the next example. Then choose a value of r so that Cr < 1/10. Define
:= r/ n, and consider the lattice of points of the form

xk,` := (k, `) .
There are finitely many such points in our box B, and moreover. If we take the collection of the balls
of radius r centered on these points, their union covers all of B.
Now, go through a list of all of these points (preferably using a computer), and for each one,
determine whether or not |fj (xk,l )| Cr for each j = 1, . . . , n. If not, there is no solution in a ball
of radius f about xk,` , and we discard that point from our list.
We are then left with a short list of of points xk,` for which there can be a solution of f (x) = 0
nearby. We would then run Newtons method for each of these points.
Of course, the choice of the value 1/10 for our selection of r through Cr < 1/10 may not be
suitable for generating a short list for any given problem, though it will not be bad in practice. If
a large fraction of the points in the grid are kept and not discarded, then one should go back and
lower the value of r by at least an order of magnitude.
Later in the course, we will have more to say about how to avoid trial and error, but in practice,
this simple scheme works very well for finding all of the solutions of f (x) = 0 that lie in any given
closed, bounded, conves set B, and in particular, determining how many such solutions there are.
Finally there is the matter of stopping Newtons method. Again, we will return to this later, but
you will observe the following in the exercises: Once you get one or two digits after the decimal point
correct, the number of correct digits approximately doubles with each iteration. Since 220 > 106 ,
it follows that you will have about 6 correct digits in 20 iterations provided you picked a starting
point, perhaps using one of the methods described above, that has one or two digits to the right of
the decimal point correct.
174
4.5
The structure of linear transformations from Rn to Rm
4.5.1
The Fundamental Theorem of Linear Algebra
As we have seen, the derivative at x0 of a differentiable function from Rn to Rm specifies a linear

transformation transformation from Rn to Rm that is a particularly good approximation to the
original function near x0 . This approximation is useful because linear transformations have a fairly
simple structure at least compared to the wild and untamed category of non-linear transfomrations.
In this section we prove several basic facts about the general structure of linear functions.
Let A be an m n matrix, and let f denote the corresponding linear transformation f (x) = Ax.
If we are trying to solve the equation Ax = b, two questions arise:
(1) For which vectors b Rm does at least one solution exist?
(2) When at least one solution does exist, how many are there? When is there a unique solution?
Definition 58 (Range and Nullspace). Let A = [v1 , . . . , vn ] be an m n matrix. The range of A,
Ran(A), is the set of vectors in b Rm such that there exists at least one x Rn such that Ax = b.
The nullspace of A, Null(A), is the set of all vectors x Rn such that Ax = 0.
Note that Ran(A) is a subspace of Rm . Indeed, if x and y are any two vectors in Ran(A), there
are by definition vectors v and w in Rn so that Av = x and Aw = y. Then if a and b are any two
numbers,
ax + by = aAv + bAw = A(av + bw) Ran(A) .
Thus, Ran(A) possesses the defining property of a subspace.
Likewise, Null(A) is a subspace of Rn . Indeed, if x and y are any two vectors in Null(A), and A
and b are any two numbers,
A(ax + y) = aAx + bAy = a0 + b0 = 0 .
Hence ax + y Null(A), and so Null(A) possesses the defining property of a subspace.
These two subspaces are the key to answering the questions raised just before Definition 58.
Note that, directly from the definition,
Ax = b has at least one solution for each b Rn if and only if Ran(A) = Rn , which is the case if
and only if dim(Ran(A)) = n.
Also, let x1 and x2 be two solution of Ax = b. Then
A(x1 x2 ) = b b = 0 ,
so that x1 x2 Null(A). Thus if Null(A) = {0}, then x1 = x2 , and whenever Ax = b has a
solution, the solution is unique. On the other hand, if there exists a non-zero vector w Null(A),
and if Ax = b has at least one solution x0 , then for all t R,
A(x0 + tw) = Ax0 + tAw = b + t0 = b ,
and hence there are infinitely many solutions. We conclude:
4.5. THE STRUCTURE OF LINEAR TRANSFORMATIONS FROM RN TO RM
175
When b Rn is such that Ax = b has at least one solution, then this solution is unique if and only
if Null(A) = {0}, which is the case if and only if dim(Null(A)) = 0. Otherwise there are infinitely
many solutions.
Though Ran(A) is a subspace of Rm and Null(A) is a subspace of Rn , and m may be different
from n, it turns out that the dimensions of Ran(A) and Null(A) are related in a way that depends
on n alone:
Theorem 50 (The Fundamental Theorem of Linear Algebra). Let A be an m n matrix. Then
dim(Null(A)) + dim(Ran(A)) = n .
(4.77)
We postpone the proof, and first explain an application. Recall that Theorem 40, whose proof
we have postponed, says that a linear transformation from Rn to Rn is one-to-one if and only if it
maps Rn ono Rn . This is an easy consequence of Theorem 50:
Proof of Theorem 40: let f be a linear transformation from Rn to Rn . Let A be the corresponding
n n matrix. By what we explained above, the linear transformation x 7 f (x) = Ax transforms Rn
onto Rn if and only if dim(Ran(A)) = n, and it is one-to-one if and only if dim(Null(A)) = 0. Then
by (4.77), dim(Ran(A)) = n dim(Null(A)) = 0.
The formula (4.77) has many other important applications in the theory of equations involving
linear transformations. Therefore, the dimensions in this formula have names:
Definition 59 (Nullity and rank). Let A be an m n matrix. The nullity of A, nullity(A), is given
by nullity(A) = dim(Null(A)). The rank of A, rank(A), is given by rank(A) = dim(Ran(A)).
In the final subsection, we shall see how to compute the rank (and hence the nulllity), and to
find all solutions of equations of the form Ax = b. First, we turn to the proof of Theorem 50.
Our proof of (4.77) will use the notion of linear independence. We have defined the dimension
of a non-zero subspace V of Rn to be the maximal size of an orthonormal subset {u1 , . . . , uk } V .
We now define a class of sets of vectors more general more general than orthonormal sets, but which
can also be used to determine dimension.
4.5.2
Linear independence and Gram-Schmidt orthonormalization
Definition 60 (Linear independence). A set {v1 , . . . , vk } Rn is linearly independent in case

k
X
the only linear combination
xj vj that equals 0 is the trivial linear combination in which each
j=1
coefficient is zero. That is, if

k
X
xj v j = 0
xj = 0
for each j = 1, . . . , k .
(4.78)
j=1
Notice that if {v1 , . . . , vk } is linearly independent, then it is impossible to find {x2 , . . . , xk } so

k
k
X
X
that v1 =
xj vj since then defining x1 := 1, we would have
xj vj = 0, but x1 6= 0. Hence
j=1
j=1
176
the direction specified by v1 cannot be achieved using any linear combinations of the other vectors
in the set. Permuting the indices, this argument applies to each vector in the set. In this sense, the
vectors are all independent.
Next notice that any orthonormal set {u1 , . . . , uk } is linearly independent: Since
2

k
k
X

X
=

x
u
x2j ,
j
j

j=1
j=1
(4.78) is true.
Not every linearly independent set is orthonormal: Consider for example {v1 , v2 } := {(1, 1, 0), (1, 0, 1)}
3
R . Then
x1 v1 + x2 v2 = (x1 + x2 , x1 , x2 ) ,
and this equals 0 if and only if x1 = x2 = 0. However, v1 and v2 are not orthogonal. Nonetheless,
we have the following theorem:
Theorem 51 (Gram-Schmidt Orthonormalization). Let {u1 , . . . , uk } be a linearly independent subset
of Rn . Then there exists an orthonormal subset {u1 , . . . , uk } of Rn such that:
(1) For each j = 1, . . . , k, uj is a linear combination of the vectors in {v1 , . . . , vj }.
(2) For each j = 1, . . . , k, vj is a linear combination of the vectors in {u1 , . . . , uj }.
Example 74. Consider the linearly independent set {v1 , v2 } := {(1, 1, 0), (1, 0, 1)} introduced above.
Let us define u1 by normalizing v1 : We compute kv1 k = 2 and define

u1 :=
1
1
v1 = (1, 1, 0) .
kv1 k
2
Now we want to use v2 to build a direction that is orthogonal to u1 , and we want to do it in a way
that works in any dimension, so we will not use a cross product. Instead, we define w2 by subtracting
off from v2 its component that is parallel to u1 : That is,
w2 := v2 (v2 u1 )u1 .
(4.79)
1
1
Computing, we find v2 u1 = (1, 0, 1) ((1, 1, 0) = , and hence
2
2
1
1
w2 = (1, 0, 1) (1, 1, 0) = (1, 1, 2) .
2
2
Note that not only is w2 orthogonal to u1 , since u1 is a multiple of v1 , it is also a linear combination
of v1 and v2 . We normalize it to define u2 : In fact, we may as well ignore the 1/2, and normalize
(1, 1, 2) since it is the direction of w2 that concerns us. We compute k(1, 1, 2)k = 6, and so
1
u2 := (1, 1, 2) .
6
Since u2 is a multiple of w2 , and since w2 is a linear combination of v1 and v2 , so is u2 . This gives
us our orthonormal set {u1 , u2 }. We have already explained why it has property (1) in Theorem 51.
To see it also has property (2), note that by construction v1 is a multiple of u1 , and by (4.79),
v2 = w2 + (v2 u1 )u1 ,
and w2 is a multiple of u2 .
177
The procedure applied in the example easily extends to larger sets of vectors: We build the
set {u1 , . . . , uk } vector by vector out of {v1 , . . . , vk } in a recursive procedure: Start by normalizing
v1 to obtain u1 . Then, when for ` < k, {u1 , . . . , u` } is defined, form w`+1 by subtracting off the
components parallel to the directions already defined:
w`+1 := v`+1
`
X
(v`+1 uj )uj ,
j=1
1
w`+1 . Of course, this normalization is possible
kw`+1 k
if and only if kw`+1 k 6= 0. As we shall see in the proof of Theorem 51 that follows, it is the
and then normalizing this to produce u`+1 =
assumption of linear independence, together with the properties (1) and (2) in the theorem, which
ensures that we will never divide by zero in the procedure of subtracting off parallel components and
then renormalizing.
Proof of Theorem 51: Since {u1 , . . . , uk }, v1 6= 0. Define u1 = (1/kv1 k)v1 . If k = 1, we are
finished: {u1 } is an orthonormal set, and v1 and u1 are multiples of one another.
If k 2, we proceed to define u2 as follows: First, define
w2 := v2 (v2 u1 )u1 .
Then w2 u1 = v2 u1 (v2 u1 )u1 u1 = 0. Hence w2 is orthogonal to u1 . Next, since u1 is a multiple
of v1 , w2 is a linear combination of the vectors in {v1 , v2 }, and the coefficient of v2 is not zero. Since
{v1 , v2 } is linearly independent, w2 6= 0. Now we normalize w2 : Define u2 = (1/kw2 k)v2 . Then
{u1 , u2 } is orthonormal.
Since w2 is a linear combination of the vectors in {v1 , v2 }, and since u2 is a multiple of w2 , u2
is a linear combination of the vectors in {v1 , v2 }. Also, since
v2 = w2 + (v2 u1 )u1
and since w2 is a multiple of u2 , v2 is a linear combination of the vectors in {u1 , u2 }. If k = 2 we
are now done.
We now proceed inductively. Suppose that for ` < k, we have found an orthonormal set
{u1 , . . . , u` } such that (1) and (2) are true for each j `. Define
w`+1 := v`+1
`
X
(v`+1 uj )uj .
j=1
Note that for i `,

w`+1 ui = v`+1 ui
`
X
(v`+1 u j )(uj ui ) = v`+1 ui v`+1 ui = 0 .
j=1
Thus, w`+1 is orthogonal to each vector in {u1 , . . . , u` }.

Next, since each uj is a linear combination of the vectors in {v1 , . . . , v` }, w`+1 is a linear
combination of the vectors in {v1 , . . . , v`+1 }, and the coefficient of v`+1 is not zero. Then since
{v1 , . . . , v`+1 } is linearly independent, w`+1 6= 0. Thus we may normalize w`+1 to define u`+1 :=
(1/kw`+1 k)w`+1 . Then {u1 , . . . , u`+1 } is orthonormal.
178

It remains to show that v`+1 is a linear combination of the vectors in {u1 , . . . , u`+1 }, and that
u`+1 is a linear combination of the vectors in {v1 , . . . , v`+1 }. The latter is clear since we have already
seen that w`+1 is a linear combination of the vectors in {v1 , . . . , v`+1 }, and u`+1 is a multiple of
w`+1 . Finally, we have from the definition of w`+1 that
v`+1 = w`+1 +
`
X
(v`+1 uj )uj .
j=1
Then since w`+1 is a multiple of u`+1 , v`+1 is a linear combination of the vectors in {u1 , . . . , uj+1 }.
This completes the proof of the inductive step.
Theorem 51 and its proof are very important: The proof is important because is gives a procedure
for constructing orthonormal sets in any dimension. Earlier in the text, working in R3 , we used the
cross product for this purpose, but the cross product is special to three dimensions. First, let us give
an important application of Theorem 51 itself:
Theorem 52 (Maximal linear independent sets and dimension). Let V be a subspace of Rn , and
suppose that {v1 , . . . , vk } is a linearly independent set in V that is maximal in the sense that it is
not a proper subset of any other linearly independent set in V . Then the dimension of V is k, and
for every vector x V , there is a unique vector (y1 , . . . , yk ) Rk such that
x=
k
X
yj vj .
j=1
Proof: By Theorem 51, there exists an orthonormal set {u1 , . . . , uk } such that each uj is a linear
combination of the vectors in {v1 , . . . , vk }, and such that each vj is a linear combination of the
vectors in {u1 , . . . , uk }.
It follows that each uj V since uj a linear combination of the vectors in {v1 , . . . , vk }. Thus,
{u1 , . . . , uk } is an orthonormal subset of V .
If {u1 , . . . , uk } were not an orthonormal basis of V , there would exist a a non-zero vector u V
that is orthogonal to each uj . But then since each vj is a linear combination of the vectors in
{u1 , . . . , uk }, u is orthogonal to each vector in {v1 , . . . , vk }. It would then follow that {v1 , . . . , vk , u}
would be a linearly independent subset of V . But this is impossible since {v1 , . . . , vk } is maximal.
Hence {u1 , . . . , uk } is an orthonormal basis of V , and dim(V ) = k.
Next, {u1 , . . . , uk } is an orthonormal basis of V , for all x V ,
x=
k
X
(x uj )uj .
j=1
But then since each uj is a linear combination of the vectors in {v1 , . . . , vk }, there exist numbers
Pj
Pj
y1 , . . . , yk such that x = j=1 yj vj . Finally, if z1 , . . . , zk are such that x = j=1 zj vj , then
0=xx=
j
X
j=1
yj v j
j
X
j=1
j
X
zj vj =
(yj zj )vj .
j=1
Since {v1 , . . . , vk }, yj zj = 0 for each j, Thus, there is exactly one way to write each x V as a
linear combination of the vectors in {v1 , . . . , vk }.
179
Proof of Theorem 50: Let k = dim(Null(A)). Let {u1 , . . . , un } be an orthonormal basis of Rn in

which the first k vectors are an orthonormal basis for Null(A), and the last n k are an orthonormal
basis for Null(A) . (If k = 0, Null(A) = {0}, and so Null(A) = Rn ; in this case any orthonormal
basis of Rn will do.)
For any x Rn we have x =
n
X
(x uj )uj . Then since Auj = 0 for j k,
=1
Ax =
n
X
(x uj )Auj .
=k+1
Thus every vector in Ran(A) is a linear combination of the vectors in {Auk+1 , . . . , Aun }.
We now claim that the set {Auk+1 , . . . , Aun } is linearly independent. To see this, suppose that
n
X
for some numbers yk+1 , . . . , yn ,
yj Auj = 0. Then
j=k+1
0=
n
X
yj Auj = A
j=k+1
so that
n
X
n
X
yj uj ,
j=k+1
yj uj Null(A). Since this vector is a linear combination of vectors in Null(A) , it is
j=k+1
orthogonal to itself, and is therefore zero. But

2
X

n
X
n

0=
y
u
=
yj2 ,
j j

j=k+1

j=k+1
and so each yj = 0. This proves the linear independence.
We now claim that {Auk+1 , . . . , Aun } is not a proper subset of any linearly independent subset
of Ran(A). Indeed, since we have already seen that every vector in Ran(A) can be written as a linear
combination of the vectors in {Auk+1 , . . . , Aun }, adding in any other vector would break the linear
independence. By Theorem 52, dim(Ran(A)) = n = k.
4.5.3
Computing rank and solving linear equations
The following theorem is the basis of a simple and effective way to compute the rank of a matrix:
Theorem 53 (Rank and rows). Let A be an m n matrix. Let {r1 , . . . , rm } be its rows. Then
rank(A) = dim(Span({r1 , . . . , rm })) .
(4.80)
Proof: By Theorem 39, Ax = 0 if and only if rj x = 0 for each j = 1, . . . , n, which is the case if
and only if x is orthogonal to all of the rows of A. That is,
Null(A) = {r1 , . . . , rm } .
Consequently, by Theorem 15,
Null(A) = ({r1 , . . . , rm } ) = Span({r1 , . . . , rm }) .
(4.81)
180
By the Fundamental Theorem of Linear Algebra, rank(A) = n dim(Null(A)). By (4.81) and

Theorem 14, which says that for any subspace V of Rn , dim(V ) + dim(V ) = n, (4.80) follows.
Next, we observe that if A has a particular structure, the dimension of the span of the rows of
A is easy to determine:
Definition 61 (Row echelon form). Let U be an m n matrix. Let j(i) denote the position of the
first non-zero entry in the ith row of A, provided there is one, and set j(i) = n + 1 otherwise. Then
U is in row-echelon form if and only for each i = 1, . . . , m 1,
j(i) j(i + 1)
with strict inequality whenever j(i) n. The first non-zero entries in the non-zero rows are called
pivotal entries, and the columns that contain them are called pivotal columns.
A matrix in row-echelon form has the
following staircase appearance
0 0
,
0 0 0
0 0 0 0 0
where denotes an entry that is definitely not zero, and denotes an entry that can be anything.
The entries are the pivotal entries, and the columns that contain them are the pivotal columns.
The point of the definition is that, even without knowing the actual entries, we can see that the
three non-zero rows of this matrix are linearly independent:
Call the rows s1 , s2 and s3 . If x1 s1 + x2 s2 + x3 s3 = 0, then first of all x1 = 0, since the second
entry of x1 s1 + x2 s2 + x3 s3 is x1 times the bullet entry in s1 . So x1 = 0, But then x2 s2 + x3 s3 = 0.
Next, the fourth entry in x2 s2 +x3 s3 is x2 times the bullet entry in s2 . Hence x2 = 0, and x3 s3 = 0.
Since s3 6= 0, it follows that x3 = 0. Hence x1 = x2 = x3 = 0 whenever x1 s1 + x2 s2 + x3 s3 = 0. Thus
{s1 , s2 , s3 } is linearly independent.
Clearly, the span of the rows of U is the span of {s1 , s2 , s3 }, and so {s1 , s2 , s3 } is a maximal
linear independent set in span({s1 , s2 , s3 }). Thus, rank(U ) = 3. The following general result is now
evident:
Theorem 54 (Rank and echelon form). Let U be an m n matrix in row-echelon form. Then the
rank of U is the number of non-zero rows of U .
How is this useful if A is not in row-echelon form? There is a simple step-by-step procedure for
transforming any m n matrix A into an m n matrix U that is in row-echelon from, and such
that at each step of the transformation, the rank is unchanged. So to compute the rank of A, make
the sequence of transformations, and when you arrive at the row-echelon form matrix U , count the
non-zero rows. This number if the rank of U , and since the transformation process did not change
the rank at any step, it is also the rank of A. This gives us the rank of A.
At each step of the transformation process, we apply one of the following operations on the set
{r1 , . . . , rm } of the rows of the matrix:
181
(1) Changing the order of the listing; in particular, swapping two of the rows.
(2) Multiplying any of the rows by a non-zero number.
(3) Adding a multiple of one row to another.
The key point is that none of there operations change the span of the rows, and hence none change
the nulls space, and hence none change the rank. These operations are called called elementary row
operations.
It is evident that after preforming either of the first two operations, we can form the exact same set
of linear combinations as we could before. Hence these operations do not affect Span({r1 , . . . , rm }),
and hence by Theorem 53, they do not affect the rank of A.
After doing the third operation, one of the new rows is a linear combination of the old rows.
Since linear combinations of linear combinations are linear combinations, every linear combination
of the new rows is a linear combination of the old rows. But since the operation can be undone by
an operation of the same type only changing the sign of the multiple every linear combination
of the old rows is a linear combination of the new rows. Hence, the third operation does not affect
Span({r1 , . . . , rm }), and hence does not affect rank(A).
We now claim that using a finite number of elementary row operations, we can transform any
m n matrix A into a matrix U in row-echelon form, thus revealing the rank of A. An example will
make the basis of this claim evident.
1 2 2 0
Example 75. Let A = 1 2 1 1
2
.
2 4 3 1 3
Subtracting the first row from the second row, and then twice the first row from the third row
transforms A as follows:
0 0
0 0
1
.
1
This has cleaned out the first column, and in the process, the second column as well. But we chose
the multiples to clean out the first column. We do not yet have the staircase pattern we seek: The
second and third rows both begin in the third column. To clean up the third column, subtract the
second row from the third:
1 2
0 0
0 0
0 0
0 0
1
:= U .
0 0
The result, U , is in row-echelon form. Since it has two non-zero rows, we see that the rank of U is
2, and hence the nullity of U is 3. But since the row operations do not change the rank or nullity,
we see that rank(A) = 2 and nullity(A) = 3.
Actually, there is much more one can learn by using row operations to transform a matrix into
row-echelon form. Indeed, let A be an m n matrix and b Rm . For each x Rn , let (x, 1)
182
denote the vector in Rn+1 whose first n entries are the entries of x and whose whose (n + 1)st entry
is 1. Let [A|b] be the m (n + 1) matrix whose first n columns are the columns of A, an whose
(n + 1)st column is b. Then [A|b] is called the augmented matrix associated to the equation Ax = b.
The point of these definitions lies in the observation that
Ax = b
[A|b](x, 1) = 0
(x, 1) Null([A|b]) .
Now suppose we apply a sequence of row operations and transform [A|b] into a matrix [U |c] in
row-echelon form. Then since row operations do not change the nullspace,
(x, 1) Null([A|b])
(x, 1) Null([U |c])
and hence
Ax = b
Ux = c .
Since [U |c] is in row-echelon form, U is in row echelon form, and consequently, it is easy to determine
whether U x = c has a solution, and when it does, to find the general solution. An example will make
this clear.
Example 76. Let A =

1
2
and let b = (4, 3, 7). Then [A|b] = 1 2 1 1
1 3
2 4 3 1
elementary row operations that we made in Example 75,
1 2
2 0 1
4
1 4
2 3
0 0 1 1 1 1 .
3
.
7
2 4 3
Now making the same sequence of
1 2 2 0
1 2 1 1
2 4 3 1 3
Hence the equation

Ax = b has the same solutions set as the equation U x = c where U =
2
2 0 1
0 0 1 1 1 and c = (4, 1, 0). Writing U x = vc out explicitly, we get
0 0
0 0 0
x1 + 2x2 + 2x3 + 0x4 + x5
x3 + x4 + x5
The last equation is satisfied for any choice of x4 and x5 provided

x3 = 1 + x4 + x5 .
Substituting this into the first equation,
x1 + 2x2 + 2x3 + 0x4 + x5 = 4
x1 + 2x2 + 2(1 + x4 + x5 ) + x5 = 4
This is satisfied for all choices of x2 , x4 and x5 provided

x1 = 2 2x2 2x4 3x5 .
x1 + 2x2 + 2x4 + 3x5 = 2 .
183
Thus the general solution of U x = c, and hence of Ax = b, is

(2 2x2 2x4 3x5 , x2 , 1 + x4 + x5 , x4 , x5 ) = (2, 0, 1, 0, 0)+
x2 ( 2, 1, 0, 0, 0) + x4 ( 2, 0, 1, 1, 0) + x5 ( 3, 0, 1, 0, 1) .
Since the variables x2 , x4 and x5 are free and can be assigned any values at all, one solution is
obtained by setting them all equal to zero, leaving x = (6, 0, 1, 0, 0). Indeed, a simple computation
give us
A(2, 0, 1, 0, 0) = 2(1, 1, 2) + (2, 1, 3) = (4, 3, 7) ,
as claimed.
We can rename the three free variables, and consider them as parameters in a parameterization
of the solutions set of Ax = b, given by
x(r, s, t) = (2, 0, 1, 0, 0) + r( 2, 1, 0, 0, 0) + s( 2, 0, 1, 1, 0) + t( 3, 0, 1, 0, 1) .
Thus, our row reduction procedure leads us to the general solution of Ax = b. We can go further,
and, for example, find bases for Null(A) and Ran(A) from the row reduction we have computed. Lets
us start with Null(A).
Applying A to both sides, since x(r, s, t) is a solution, we get
b = Ax(r, s, t)
A(2, 0, 1, 0, 0) + rA( 2, 1, 0, 0, 0) + sA( 2, 0, 1, 1, 0) + tA( 3, 0, 1, 0, 1)
= b + rA( 2, 1, 0, 0, 0) + sA( 2, 0, 1, 1, 0) + tA( 3, 0, 1, 0, 1) .

It follows that for all r, s and t,
rA( 2, 1, 0, 0, 0) + sA( 2, 0, 1, 1, 0) + tA( 3, 0, 1, 0, 1) = 0 ,
and thus that each of the vectors in
{ ( 2, 1, 0, 0, 0) , ( 2, 0, 1, 1, 0) , ( 3, 0, 1, 0, 1) }
(4.82)
is in Null(A). Since
r( 2, 1, 0, 0, 0) + s( 2, 0, 1, 1, 0) + t( 3, 0, 1, 0, 1) = ( 2r 2s 3t, r, s + t, s, t),
Since the second entry of the sum is r, the fourth is s and the fifth is t, the sum cannot be zero unless
r = s = t. Thus the set in (4.82) is linearly independent. Since the dimension of the nullspace of A
is 3, this is a maximal linearly independent set. Hence the set in (4.82) is a basis for Null(A).
Now let us find a basis for Ran(A) Notice that the free variables are those corresponding to
columns without pivots: x2 , x4 and x5 are free, and columns 2, 4 and 5 have no pivots in them after
the row-reduction. Since these variables can be assigned the value zero, the corresponding columns
are not needed in order to solve the equation. By definition Ax = b has a solution when
[v1 , v2 , v3 , v4 , v5 ](x1 , x2 , x3 , x4 , x5 ) =
5
X
j=1
xj v j = b
184
Whenever this is the case, there is a solution (y1 , y2 , y3 , y4 , y5 ) in which y2 = y4 = y5 = 0. Thus,

b = y1 v1 + y3 v3 for some (uniquely determined) y1 and y2 . Thus, {v1 , v2 } spans Ran(A), which is
the span of {v1 , v2 }. Since the dimension of the range of A is 2, this is a basis for Ran(A). Notice
that our basis for Ran(A) was the set of columns in the original matrix A that ended up as pivotal
columns after the row reduction. A bit of thought shows that this will always be the case.
4.6
Exercises
4.1 Let v1 = (1, 1) and v2 = (0, 1). Let f : R2 R be differentiable, and suppose that for some
x0 R2 , v1 f (x0 ) = 2 and v2 f (x0 ) = 2. For v = (2, 3), compute v f (x0 ).
4.2 Let v1 = (1, 1) and v2 = (3, 1). Let f : R2 R be differentiable, and suppose that for some
x0 R2 , v1 f (x0 ) = 2 and v2 f (x0 ) = 3.
For v = (1, 1), compute v f (x0 ).
4.3 Let v1 = (1, 1, 1), v2 = (0, 1, 1) and v3 = (0, 0, 1). Let f : R3 R be differentiable, and suppose
that for some x0 R3 ,
v1 f (x0 ) = 5
v2 f (x0 ) = 3
and
v3 f (x0 ) = 2 .
For v = (1, 2, 3), compute v f (x0 ).

4.4 Let f : R2 R be given by f (x, y) = x2 y + yx xy 2 . Let x(t) b given by x(t) = (t, t2 ) Compute

d
f (x(t)) .
dt
t=1
xy
.
(1 + x2 + y 2 )2
(a) Find all of the critical points of f , and find the value of f at each of the critical points.
4.5 Let f (x, y) =
(b) Does f have a maximum value? Explain why or why not. If it does, find all points at which the
value of f is maximal; i.e, find all maximizers.
(c) Does f have a minimum value? Explain why or why not. If it does, find all points at which the
value of f is minimal; i.e, find all minimizers.
4.6 Let f : R2 R be given by f (x, y) = x2 y + yx xy 2 .
(a) Compute the gradient of f , and find all critical points of f ..
(b) Find the equation of the tangent plane to the graph f at the point (1, 1).
4.7 Let f (x, y) 3xy x3 y 3 . Find all points (x, y) at which the tangent plane to the graph of f
is orthogonal to the line parameterized by t(3, 3, 1).
4.8 Let f : R2 R be given by f (x, y) = x3 + y 3 + 3xy.
(a) Compute the gradient of f , and find all points (x, y) at which the tangent plane to the graph of
f is horizontal.
(b) Find the equation of the tangent plane to the graph of f at the point (1, 2).
4.6. EXERCISES
185
(c) If you were standing at the point (1, 1) and wanted to climb uphill as directly as possible, in
which compass direction would you head? For purposes of this question, take the direction e1 to be
East, and the direction e2 to be North.
4.9 Let v1 = (1, 1, 1), v2 = (0, 1, 1) and v3 = (0, 0, 1). Let f : R3 R be differentiable, and suppose
that for some x0 R3 ,
v1 f (x0 ) = 5
v2 f (x0 ) = 3
and
v3 f (x0 ) = 2 .
Find a right-handed orthonormal basis {u1 , u2 , u3 } of R3 such that u1 is parallel to f (x0 ).

"
#
5 2
4.10 Let A :=
.
2 1
(a) Compute det(A) and the matrix inverse of A.
(b) Find all solutions of the equation Ax = (1, 2).
#
"
1 2
.
4.11 Let A :=
1 1
(b) Find all solutions
4.12 Let A :=
1
1
of the equation Ax = (3, 1).
0 3
2 1
.
1

(b) Find all solutions of the equation Ax = (1, 2, 3).
1 2 1
4.13 Let A :=
2 1 1 .
1 1 2
(b) Find all solutions of the equation Ax = (4, 0, 8).
4.14 Let f (x, y) = x2 y + (x 1)(y 1)2 .
(a) Find the equations for the tangent planes to the graph of z = f (x, y) at x1 = (2, 1) and at
x2 = (1, 1).
(b) Parameterize the line that is the intersection of the two planes found in (a).
(c) Let x0 = (3, 0, 5). Compute the distance from x0 to the line found in (b), and from x0 to the
second tangent plane found in (a). The distance to the plane should be smaller than the distance to
the line. Explain why.
4.15: Let f (x, y) and g(x, y) be given by
f (x, y) = x2 y 4y + x3
and
g(x, y) = 4x2 + 4y 2 1 .
186

Define the function f from R2 to R2 by
f (x) = (f (x), g(x)) .
(a) Let x0 = (1/2, 0). Compute f (x0 ) and the Jacobian of f at x0 ; i.e., [Df (x0 )].
(b) Use x0 as a starting point for Newtons method for solving f (x) = 0, and find x1 , the next step.
(c) Evaluate f (x1 ). Comment on your result - how many solutions are there, and how close is x1 to
one of them?
4.16 Let f (x) = (f (x), g(x)) where f (x, y) = x3 + xy, and g(x, y) = 1 4y 2 x2 . Let x0 = e1 .
(a) Compute [Df (x)] and [Df (x0 )].
(b) Use x0 as a starting point for Newtons method, and compute the next approximate solution x1 .
(c) Evaluate f (x1 ), and compare this with f (x0 ).
4.17 Let f (x) = (f (x), g(x)) where f (x, y) = x + y 3, and g(x, y) = x2 + y 2 18. Compute
f (x0 ) for x0 = (3, 3). Does this look like a reasonable starting point? Compute [Df (x0 )]. What
happens if you try to use x0 as your starting point for Newtons method?
4.18: Let f (x, y) and g(x, y) be given by
f (x, y) = (x + 1)2 + (y + 1)2 4
and
g(x, y) = 4(x 1)2 + (y 1)2 5 .
(a) The equation f (x, y) = 0 describes a circle. The equation g(x, y) = 0 describes an ellipse. Sketch
a plot of the circle and the ellipse, and determine how many solutions there are of the system
f (x, y) := (f (x, y), g(x, y)) = 0 .
Note: It is possible to exactly solve this system by algebraic means, but that is not what is being
asked for here.
(b) Your sketch should show one solution of the system in the lower right hand quadrant not too far
from (1, 1). Use x0 = (1, 1) as a starting point, and apply one step of Newtons method to find
x1 , a better approximate solution, and compute f (x1 ) and g(x1 ).
4.19 Let f (x) = (f (x), g(x)) where f (x, y) = sin(xy) x, and g(x, y) = x2 y 1. Let x0 = (1, 1).
(a) Compute [Df (x)] and [Df (x0 )].
(b) Use x0 as a starting point for Newtons method, and compute the next approximate solution x1 .
(c) Evaluate f (x1 ), and compare this with f (x0 ).
(d) How many solutions of this system are there in the region 2 x 2 and 0 y 10? Compute
each of them to 10 decimal places of accuracy using a computer, of course.
4.20 Let A be a 3 3 matrix whose three rows are r1 , r2 and r3 , and whose three columns are v1 ,
v2 and v3 , so that
r1
A = [v1 , v2 , v3 ] =
r2 .
r3
4.6. EXERCISES
187
Show that
det(A) = r1 r2 r3 .
That is, show that the triple product of the rows of a 3 3 matrix is always equal to the triple
product of it columns, which, by definition, is the determinant.
4.21 Show that for all a, b and c in R3 ,
(b c) [(c a) (a b)] = |a (b c)|2 .
4.22 Let {v1 , . . . , vk } be a set of k vectors in Rn . Show that {v1 , . . . , vk } is linearly independent if
and only if nullity([v1 , . . . , vk ]) = 0.
1 2
4.23 Let A be the matrix A = 2 1

3
3
0
3
1
.
3
(a) Find a 3 4 matrix U in row-echelon form such that Null(U ) = Null(A).

(b) Compute rank(A) and nullity(A).
(c) Find a maximal linearly independent set of vectors in Ran(A), and then apply the Gram-Schmidt
orthonormalization procedure to find an orthonormal basis of Ran(A).
(d) Find an equation for Ran(A). Does the vector (1, 2, 3) belong to Ran(A)? If so, find all solutions
of Ax = (1, 2, 3). If not, explain why not.
188
Chapter 5
THE IMPLICIT FUNCTION

THEOREM AND ITS
CONSEQUENCES
5.1
Horizontal slices and contour curves
In previous chapters, we have considered vertical slices of the graph of z = f (x, y). We can gain a
new perspective by considering horizontal slices.
Consider once again the mountain landscape that we considered earlier in connection with
vertical slices:
f (x, y) =
3(1 + x)2 + xy 3 + y 2
1 + x2 + y 2
(5.1)
Suppose that a dam is built, and this landscape is flooded, up to an altitude 0.5 in the vertical
distance units. This produces a lake that is shown below, in a top view; i.e., an aerial image:
The other lines on the land are the lines at other constant altitudes, specifically x = 1.5, z = 2.5,
z = 3.5 and so on. On a topographic map, these curves are called contour curves. Here is a sort of
side view showing the lake as a horizontal slice through the graph z = f (x, y) at height z = 0.5:

189
190
CHAPTER 5. THE IMPLICIT FUNCTION THEOREM AND ITS CONSEQUENCES
If the water level is raised further, say to the altitude z = 1.5, everything will be flooded up to
the next contour curve:
Comparing with the first picture, you clearly see that everything has been flooded up to the
z = 1.5 contour curve. The isthmus joining the two tall hills is now submerged, and the two regions
of the lake in the first graph have merged.
If you walked along the lake shore, your path would trace out the contour curve at z = 1.5 in
the first picture.
Here is a side view showing the lake at this level. It shows it as a horizontal slice through the
graph z = f (x, y) at height z = 1.5:
5.1. HORIZONTAL SLICES AND CONTOUR CURVES
191
If the water level is raised further, to the height z = 2.5, the shore line moves up to the next
contour line. Now a walk along the shoreline would trace out the path along the x = 2.5 contour line
in the first picture. Here is the top view showing the lake at this stage:
The contour curves, which are the results of horizontal slices of the graph of z = f (x, y), tell us
a lot about the function f (x, y). This section is an introduction to what they tell us.
First, let us recall a definition from Chapter 3: Given a function f : Rn R, the set of points
x Rn satisfying satisfying
f (x) = c
is the level set of f at c In other words, the level set of f at c is the solution set of the equation
f (x) = c.
Let us consider the case n = 2 more closely. If we think of f (x, y) as representing the altitude at
the point with coordinates (x, y), then the level set of f at height c is the set of all points at which
the altitude is c. The level set at height c would be the shore line curve if the landscape were
flooded up to an altitude c.
Now, here is a very important point, whose validity you can more or less see from the pictures
we have displayed:
Under normal circumstances, the level set of f at c will be a curve in the plane, possibly with several
disconnected components.
192

It is for this reason that it is reasonable to refer to level sets as contour curves. We can plot a
number of the level sets on a common graph. A contour plot of a function f (x, y) is graph in which
level curves of f are plotted at several different altitudes c1 , c2 , c3 , . . . . You have probably seen
these on maps for hiking.
Here is a contour plot for the function mountain landscape function f (x, y) in (5.1):
3
2
y
1
x
1
2
3
5.1.1
Implicit and explicit descriptions of planar curves
How could one go about actually drawing the contour curves starting from a formula like (5.1)? That
is not so easy in general. You can see a hint of this in the convoluted form of the contour curves
plotted here. The difficulty lies here:
The description of contour curves given by the defining equation f (x, y) = c is, alas, an implicit
description. However, plotting a curve is a simple matter only when one has an explicit description.
To really appreciate this point, one has to understand the distinction between an implicit and
and explicit description of a curve. The unit circle is a great example with which to start.
Let f (x, y) = x2 + y 2 and let c = 1. Then the level set of f at height c is the set of points (x, y)
satisfying
x2 + y 2 = 1 .
(5.2)
This set, of course, is the unit circle. If we drew a contour plot of f showing the level curves at
several altitudes altitudes c1 , c2 , c3 , . . . , you would see, several concentric circles.
The equation (5.2) is the implicit equation for the unit circle. To get an explicit description, just
solve the equation (5.2) to find a parameterization of the solutions set. In the case of x2 + y 2 = 1,
we know how to do this: As t varies between 0 and 2,
( cos t, sin t)
(5.3)
traces out the unit circle in the counterclockwise direction. This is an explicit description since if you
plug in any value of t, you get a point (x, y) on the unit circle, and as you vary t, you sweep out
193
all such points. You can easily plot a large number of them and connect the dots to get a good plot
of the circle.
Definition 62 (Implicit and explicit descriptions of curves). An equation of the form f (x, y) = c
provides an implicit description of a curve. A parameterization x(t), possibly with t = x and with
y(x) given as an explicit function of x, provides an explicit description of a curve.
Once one has an explicit description, it is easy to generate a plot, just by plugging in values for
the parameter, plotting the resulting points, and connecting the dots. Passing from an implicit
description to an explicit description involves solving the equation f (x, y) = c to find an explicit
parameterization of the solutions set. Generally, that is easier said than done.
Example 77 (From implicit to explicit by means of algebra). Consider the function
f (x, y) = 2x2 2xy + y 2 .
The level curve at c = 1 for this function is given implicitly by the equation
2x2 2xy + y 2 = 1 .
This can be rewritten as y 2 2xy = 1 2x2 . Completing the square in y, we have
(y x)2 = 1 x2 .
Therefore, we can solve for y as a function of x, finding
y(x) = x
p
1 x2 .
If we take x as the parameter, evidently y has a real value only for 1 x 1. It is now easy to
plot the contour curve:
In this example, it was not so difficult passing from an implicit description to an explicit description; i.e., a parameterization, since the equation f (x, y) = 1 was quadratic in both x and y: We
know how to deal with quadratic equations.
194
Example 78 (Bernouli Lemiscate). Consider the function

f (x, y) = (x2 + y 2 )2 2(x2 y 2 ) ,
and consider its level set at 0; i.e., the set of all (x, y) such that f (x, y) = 0. Clearly, (0, 0) is one
such point. What about the others? In this case it is not so easy to solve for y as a function of x
as in the previous example. But if we pass to polar coordinates, we can easily parameterize the level
set. That is, for any non-zero point (x, y) in the plane, we let r denote the distance from (x, y) to
(0, 0), and let be the angle such that if (1, 0) is rotated counterclockwise through the angle , then
the rotated unit vector points in the same direction as (x, y).
It follows from this description that
p
r = x2 + y 2
x = r cos
and
y = r sin .
Making these substitutions, f (x, y) = 0 becomes f (r cos , r sin ) = 0, or

r4 2r2 (cos2 sin2 ) = r4 r2 cos(2) = 0 .
We already know that (0, 0), corresponding to r = 0, is one solution. For all of the others, r 6= 0,
and we may divide by r2 to obtain
r2 = cos(2) .
(5.4)
Now notice that the left hand side is always positive, but the right hand side is negative if /4 < <
3/4, or if 5/4 < < 7/4. There are evidently no solutions of (5.4 for such . On the other
hand, when /4 < < /4 or 3/4 < < 5/4, the right hand side is positive, and since r is
always non-negative by definition (it is a distance), there will be two branches of the solution set,
parameterized by
x() := (r cos , r sin ) = (
p
p
cos(2) cos ,
cos(2) sin ) .
for
3
5
<<
and
<<
.
4
4
4
4
One can now plug in values of , evaluate, plot the points, and connect the dots. The result is
The moral is that considering a different system of coordinates on the plane can make it much
easier to find an explicit description of the curve.
This curve has a name: It is the Bernouli lemiscate, where lemiscate comes form the Latin word
for ribbon.
However, changing to polar or other standard coordinate systems is not enough to solve all such
problems. In general, we are going to need to extract information on the contour curves directly from
the implicit description. Fortunately, what we have learned about gradients can help us to do this.
5.1.2
195
The tangent line to a contour curve
Let f be a differentiable function from R2 to R, and suppose that f (x0 ) = c. Then of course, the
level set consisting of points x such that f (x) = c includes x0 .
Suppose that the level set specified by f (x) = c is a curve which has a differentiable parametrization x(t) with x(t0 ) = x0 . Since, by hypothesis, for each t, x(t), f (x(t)) = c, f (x(t)) is constant, and
hence has zero derivative. Thus, by the chain rule,
0=
d
f (x(t)) = f (x(t)) x0 (t) .
dt
In particular,
f (x0 ) x0 (t0 ) = 0 ,
and hence f (x0 ) is orthogonal to x0 (t0 ). This means that the tangent line to x(t) at t0 is orthogonal
to f (x0 ).
Now let us suppose that f (x0 ) 6= 0 so that f (x0 ) does indeed define a direction. Then, since
we are considering the two dimensional case, the only directions orthognal to f (x0 ) are multiples
of (f (x0 )) , where we recall that (a, b) := ( b, a).

Thus, an equation for the tangent line is
f (x0 ) (x x0 ) = 0 ,
(5.5)
and a parameterization of the tangent line is

x0 + s (f (x0 ))
(5.6)
Example 79 (Computing a tangent line from an implicit description). Consider the implicit de1
scription of the unit circle given by the equation x2 + y 2 = 1, and the point x0 = (1, 1), which is
2
on this circle. Of course the circle can be parameterized in a differentiable way, but let us compute
the tangent line without first finding such a parameterization.
We compute f (x0 ) = 2(1, 1). Then by (5.5), f (x0 ) (x x0 ) = 0 is the equation of the
tangent line. Working this out explicitly, we find

1
2(1, 1) (x, y) (1, 1) = ( 2x 1) + 2y 1) = 0 ,
2
and so the equation of the tangent line is x + y = 2.

The point of this example is that while you need to know an explicit parameterization of the
level curve to compute x0 (t0 ), you do not need the parameterization to find the tangent line to this
curve.
Example 80 (Tangent line to a level curve). Let f (x, y) = (x2 y 2 )2 2xy, and consider x0 = (1, 2).
We compute f (x0 ) = 5.
Let us suppose that the equation f (x, y) = 5 does indeed define a smooth contour curve passing
through x0 . (We will soon see how to easily check this hypothesis, though as in the previous example,
this can be done here using polar coordinates.)
196

To compute the tangent line to this supposed level curve of f as it passes through x0 , we first
work out that

f (x, y) = (4x(x2 y 2 ) 2y, 4y(x2 y 2 ) 2x) .
Therefore, f (1, 2) = ( 16, 22), and the equation for the tangent line is
0 = ( 16, 22) (x 1, y 2) = 16(x 1) + 22(y 2) ,
or
16x + 22y = 14 .
To find the parametric form, we compute
v = x0 = (f (x0 )) = ( 16, 22) = (22, 16).

Hence the tangent line is x0 + tv = (1, 2) t(22, 16).
5.1.3
When is the contour curve actually a curve?
In Example 80, we computed the tangent line to an implicitly defined level curve under the assumption
that the equation f (x, y) = f (1, 2) did indeed define a differentiable curve through (1, 2). How can
we check this assumption without actually finding a parameterization? The following theorem gives
the answer.
Theorem 55 (Implicit Function Theorem in R2 ). Let f be a real valued function defined on an
open set U R2 , and suppose that f is continuously differentiable in U . Then for any x0 U such
that f (x0 ) 6= 0, there is an r > 0 and a continuously differentiable curve x(t), r < t < r, with
x(0) = x0 and x0 (0) 6= 0 such that
(1) For all |t| < r, f (x(t)) = f (x0 ),
and
(2) There is an s > 0 such that every x satisfying f (x) = f (x0 ) and kx x0 k < s is of the form
x = x(t)
for exactly one value of r < t < r.
Example 81 (Branches of implicitly defined curves). Let f (x, y) = x2 y 2 , and note that f (x) = 0
if and only if x = 0.
In particular, taking x0 = (1, 0), the implicit function theorem assures us that all solutions of
f (x) = f (x0 ) that are sufficiently close to x0 lie on a differentiable curve through x0 though it
leaves open the possibility that there might be other solutions farther away.
Indeed, f (x0 ) = 1, and the equation f (x, y) = 1 is the equation of an hyperbola. This has two
p
branches given by x = y 2 + 1. The branch of the hyperbola passing through x0 = (1, 0) may be
parameterized by
p
x(t) = ( t2 + 1 , t) .
197
Then, with r := 2, all solutions of f (x) = f (x0 ) with kx x0 k < r lie on this branch of the hyperbola;
i.e., the curve parameterized by x(t). However, there are other solutions, on the other branch, further
away.
We shall state and prove a more general version of the implicit function theorem, for functions
from Rn to Rm later on. For now, let us focus on what such a theorem is good for, starting with the
version stated above.
Let us see why the condition that f be continuously differentiable at x0 with f (x0 ) 6= 0
is natural. Notice that since the gradient is continuous, for some r > 0, f (x) 6= 0 as long as
kx x0 k < r.
Suppose you are standing at x0 , and want to walk along the landscape without changing your
altitude. Let us take step sizes that are small compared with r, so that we can take many steps
before reaching a point where the gradient might be zero. In which direction should you take your
first step?
Since you do not want to go either uphill or downhill, you should step in the direction
u0 :=
(f (x0 )) ,
kf (x0 )k
or the opposite of this direction. Let us take the first choice. Here we have used the hypothesis
that f (x0 ) 6= 0 to define the unit vector u0 . Without this condition, there would be no preferred
direction.
Now after taking the step, you are at a new point x1 , pretty close to x0 . By the continuity of
f (x), and our choice of the step size, f (x1 ) 6= 0. Now choose
u1 :=
(f (x1 ))
kf (x1 )k
for the direction of your second step, and so on. At each step, as long as the gradient is not zero at
that step, the gradient tells you what direction to use for your next step: It completely defines your
path. This heuristic idea can be turned into a rigorous proof. In the next two examples, we illustrate
what can go wrong when the gradient is zero.
Example 82 (A level set that is a single point). Let f (x, y) = x2 + y 2 . Then the level set of f at
height 0, which is the level set of f passing through (0, 0) is just the single point (0, 0) and not really
a curve. We could parameterize it by x(t) = 0, but then of course we would have x0 (t) = 0. Note that
198
the Implicit Function Theorem gives us a curve that has a non-zero velocity: It is a real curve, and
does not just sit there. Notice also that f (0, 0) is zero, as the Implicit Function Theorem says it
must be if the level set is not a nice curve.
Example 83 (A level curve that crosses itself). Let f (x, y) = (x2 +y 2 )2 2(x2 y 2 ) as in Example 78.
Then the level set of f at height 0, which is the level set of f passing through (0, 0), crosses the origin
twice, as you see from the plot there, in two different directions. Here there are two level directions,
and the level set of f passing through (0, 0) is not a curve, but a union of two curves. However you
parameterize it, you have to come back and cover (0, 0) twice.
Again, notice that f (0, 0) is zero, as the Implicit Function Theorem says it must be otherwise
there would be a differentiable curve covering all points on the level set near (0, 0) in a one-to-one
way, and there is not.
There is a general conclusion to be drawn from the last example:
If a contour curve of a continuously differentiable function f on R2 crosses itself at some point,
that point must be a critical point of f .
5.1.4
Points on a contour curve where the tangent line has a given slope
Consider the curve given implicitly by

x4 + y 4 + 4xy = 0 .
This is quartic, so it is not so easy to express y as a function of x (though it can be done). A good
way to graph this curve is to notice that it is the level set of f (x, y) = x4 + y 4 + 4xy at c = 0. We
can use what we have learned to find all points (x, y) on the curve at which the tangent line through
then has a given slope s. For example, when s = 0, the line is horizontal. When s = , it is
vertical. When s = 1, the tangent line runs at an angle of /4 with respect to the xaxis. To graph
the contour curve, one can find all such points, and draw a small bit if a line segment through them
at the corresponding slope. Connecting these up, one has a sketch of the curve.
Example 84 (Points on a level curve where the tangent has a given direction). Let f (x, y) =
x4 + y 4 + 4xy, and consider the level curve given in implicit form by
x4 + y 4 + 4xy = 0 .
(5.7)
We will now find all points on this level curve at which the tangent is horizontal; i.e., parallel to the
x axis.
We compute that
f (x, y) = 4(x3 + y, y 3 + x) .
Notice that the gradient is zero exactly when
x3 + y = 0
and
y3 + x = 0 .
199
The first equation says that y = x3 . substituting this into the second equation, we eliminate y and
get x9 = x. One solution is x = 0. If x 6= 0, we can divide by x and get the equation x8 = 1,
which two solutions, x = 1 and x = 1. Then from y = x3 , we see that the three points at which
f (x) = 0 are
(0, 0)
( 1, 1)
and
(1, 1) .
(5.8)
At all other points, the level set specified implicitly by (5.7) is a differentiable curve by the
Implicit Function Theorem.
Now focus on points where f (x) 6= 0, and see at which of them the tangent line is horizontal.
The tangent to the level curve is horizontal exactly when the perpendicular direction is vertical. The
perpendicular direction is the direction of the gradient, so the tangent line is horizontal only in case
the first component of the gradient is zero, and this is exactly where
y = x3 .
(5.9)
This, together with the equation for the curve, (5.7), gives us a nonlinear system of equations for
the points we seek. To solve it, use (5.9) to eliminate y from (5.7). Doing the substitution, we get
x12 3x4 = 0. This has exactly 3 solutions: x = 0, x = 31/8 and x = 31/8 . Going back to (5.9), we
can now easily find the corresponding points. They are:
(0, 0)
(31/8 , 33/8 )
and
(31/8 , 33/8 ) .
At the first of these, the gradient is zero. That is, f (0, 0) is zero, and so it has no direction at all.
At the other two, we can be sure that the level set is a differentiable curve, and that its tangent vector
is vertical.
Deciding what actually happens at (0, 0) is more subtle. For x and y very close to zero, x4 + y 4 +
4xy 4xy, and so for such x and y, the level set of f at height 0 looks pretty much like the level set
of 4xy at height zero. This is easy to find: 4xy = 0 if and only if x = 0 or y = 0. Hence this level set
consists of two lines, namely the x and y axes, as in Example 7. At this point, there is a branch of
the level set that is vertical, but also a branch that is horizontal, so we cannot properly say that the
level curve has a horizontal tangent here.
However, using the same procedure that we used to find the points of vertical tangency, we find
that the level curve of f at height 0 has a horizontal tangent exactly at
(31/8 , 33/8 )
and
(31/8 , 33/8 ) .
By doing this for a few more slopes, and connecting the points up, you could get a pretty good sketch
of the curve. (You could also parameterize it using polar coordinates.) Here is aplot of the curve:
200

1.5
1
y
0.5
1.5
0.5
0.5
1.5
x
0.5
1.5
5.2
Constrained Optimization in Two variables
An optimization problem in two variables is one in which we are given a function f (x, y), and a set D
of admissible points in R2 , and we are asked to find either the maximum or minimum value of f (x, y)
as (x, y) ranges over D.
Recall that (x0 , y0 ) D minimizes f in D in case
f (x0 , y0 ) f (x, y)
for all
(x, y) in D ,
for all
(x, y) in D .
and (x1 , y1 ) D maximizes f in D in case

f (x1 , y1 ) f (x, y)
It can be the case there is neither a maximum nor a minimum. Consider for example f (x, y) =
x + y and D = R2 . Then
lim f (t, t) =
and
lim f (t, t) = .
Also, for f (x, y) = 1/(1 + x2 + y 2 ) and D = R2 , f (x, y) 0 for all (x, y), and for any m > 0,
f (x, y) < m for all (x, y) with x2 + y 2 > 1/ m. Hence 0 is the greatest lower bound on the values of
f (x, y) as (x, y) ranges over D. Nonetheless, there is no point (x, y) in D such that f (x, y) = 0, so 0
is not a minimum value of f it is not a value of f at all.
However, as we have seen in Chapter 3, if D is a bounded and closed region, and if f is a
continuous function, then there is always at least one minimizer and at least one maximizer.
To solve an optimization problem is to find all maximizers and minimizers, if any, and the
corresponding maximum and minimum values. Our goal in this section is to explain a strategy for
doing this. As long as D is closed and bounded, and f is continuous, minimizers and maximizers will
exist, and our goal now is compute them.
Recall that if g(t) is a function of the single variable t, and we seek to maximize it on the closed
bounded interval [a, b], we proceed in two steps:
(1) We find all values of t in (a, b) at which g 0 (t) = 0. Hopefully there are only finitely many of these,
say {t1 , t2 , . . . , tn }.
5.2. CONSTRAINED OPTIMIZATION IN TWO VARIABLES
201
(2) Compute g(t1 ), g(t2 , ) . . . , g(tn ), together with g(a) and g(b). The largest number on this finite
list is the maximum value, and the smallest is the minimum value. The maximizers are exactly those
numbers from among {t1 , t2 , . . . , tn } together with a and b, at which f takes on the maximum values,
and similarly for the minimizers.
The reason this works is that if t belongs to the open interval (a, b), and g 0 (t) 6= 0, it is possible
to move either uphill or downhill while staying within [a, b] by moving a bit to the right or the
left, depending on whether the slope is positive or negative. Hence no such point can be a maximizer
or a minimizer. This reasoning does not apply to exclude a or b, since at a, taking a step to the
left is not allowed, and at b, taking a step to the right is not allowed. Thus, we have a short list of
suspects, namely the set of solutions of g 0 (t) = 0, together with a and b, and the maximizers and
minimizers are there on this list.
In drawing up this list of suspects, we are applying the Sherlock Holmes principle:
When you have eliminated the impossible, whatever else remains, however unlikely, is the truth.
When all goes well, the elimination procedure reduces an infinite sets of suspects all of the
points in [a, b] to a finite list of suspects: {t1 , t2 , . . . , tn } together with a and b. Finding the
minimizers and maximizers among a finite list of points is easy simply compute the value of f at
each point on the list, and see which ones give the largest and smallest values.
We now adapt this to two or more dimensions, focusing first on two. Suppose that D is a closed
bounded domain. Let U be the interior of D and let B be the boundary. For example, if D is the
closed unit disk
D = {x : kxk 1}
we have
U = {x : kxk < 1}
and
B = {x : kxk = 1} .
Notice that in this case, the boundary consists of infinitely many points.
In optimization problems, the big difference between one dimension and two or more dimensions
is that in one dimension, the interval [a, b] has only finitely boundary points two and there is no
problem with throwing them into the list of suspects. But in two or more dimensions, the boundary
will generally consist of infinitely many points, and if we throw them all onto the list of suspects, we
make the list infinite, and therefore useless.
We will therefore have to develop a sieve to filter the boundary B for suspects, as well as a
sieve to filter the interior U for suspects. Here is the interior sieve:
If x belongs to U , the interior of D, and f (x) 6= 0, then x is not a suspect. Recall that f (x)
points in the direction of steepest ascent. So if one moves a bit away from x in the direction f (x),
one moves to higher ground. Likewise, if one moves a bit away from x in the direction f (x), one
moves to lower ground.
Since x is in the interior U of D, it is possible to move some positive distance from x in any
direction and stay inside D. Hence, if x belongs to U , and f (x) 6= 0, there are nearby points at
which f takes on strictly higher and lower values. Such a point x cannot be a maximizer!
202

Apply the Sherlock Holmes principle: Eliminate all points x in U at which f (x) 6= 0, and the
remaining points are the only valid suspects in U . There are exactly the critical points in U the
points in U at which f (x) = 0.
The suspect list from the interior U consists exactly of the critical points in U . This is the sieve
with which we filter the interior: We filter out the non critical points.
5.2.1
Lagranges criterion for optmizers on the boundary
Next, we need a sieve for the boundary. Here we need to make some assumptions. We will suppose
first that the boundary B of D is the set of solutions of some equation
g(x, y) = 0 .
For example, when B is the unit circle, then we have
g(x, y) = x2 + y 2 1 .
We can understand the geometrical idea behind Lagranges method by looking at a contour plot
of f with the boundary curve g superimposed. For example, let us take
f (x, y) = x2 y y 2 x .
Here is a contour plot showing the circle and some contour curves of g:
Most of the contour curves are drawn in lightly, but several others, drawn in more darkly are
tangent to the boundary at some point.
These points of tangency are the ones to pay attention to. Here is why: Suppose that you are
walking along the circle. If your path is cutting across contour curves of f at a point where f is
203
not zero, then you are going uphill or downhill at that point, and you cannot be at a minimum or
maximum value on your path.
For example, suppose that you walk clockwise on the circle from (1, 0) to (0, 1), and think of
f as representing the altitude. Note that f (1, 0) = 0 and f (0, 1) = 0, so that you are at sea level
(i.e., altitude 0) at the beginning and the end of the walk. However, in the middle of the trip, you
pass through the point

and the value of f at this point is 1/ 2.
1
1
,
2
2

,
If you look back at the contour plot, it is evident that you have been cutting across contour lines
going downhill on the first part of the walk, and then cutting across them again going uphill on the
second part of the walk.
Right in the middle, when you are neither going downhill nor uphill, you cannot be cutting across
a contour curve, and you must be walking parallel to one, so that the tangent line to your path is
parallel to the tangent line of the contour curve. This is the point of minimal altitude (which is below
sea level) along your walk.
In conclusion, we can eliminate any points on the boundary where where the tangent line to the
boundary and the tangent line to the contour curve are both well defined but not parallel, because
if f 6= 0 at such a point, the path is cutting across contour curves, and headed uphill or downhill.
We note that by the Implicit Function Theorem, the tangent line to the boundary and the tangent
line to the contour curve are both well defined at all points at which f and g are non-zero.
By the Sherlock Holmes principle, the only points to consider are points where the boundary
curve is tangent to the contour curves, or where f = 0, or where g = 0.
The last two cases are expressed in terms of equations. We now convert this tangency condition
into an equation:
The tangent direction of the contour curve of f through a point is (f ) at that point. Also,
since the boundary is a level curve of g, its tangent direction at any point on the boundary curve is
given by (g) at that point.
Therefore, the tangency means that (f ) is a multiple of (g) . But then we can undo the
perp; the gradients must themselves be multiples:
f (x) = g(x) .
(5.10)
"
We can rewrite this equation with eliminated as follows: The matrix
f (x)
g(x)
determinant if an only if one row is a multiple of the other. Hence,

"
#!
f (x)
det
=0
g(x)
#
has zero
(5.11)
if and only if both f and g are non-zero and (5.10), or if either f = 0 or g = 0.

This is nice: The single equation is satisfied not only by all points at which the tangency condition
holds, but also all points at which the tangent directions (or even the contour curves themselves)
might not be defined.
204

Of course we are only interested in points x such that (5.11) is satisfied and x is on the boundary;
i.e.,
g(x) = 0 .
(5.12)
Together, (5.11) and (5.12) give us a system of two equations in two unknowns that we can solve
to find all possible candidates for minima and maxima on the boundary. This method is often called
the method of Lagrange multipliers, and presented with the equation (5.11) in the equivalent form
(5.10). This is a vector equation, so it gives two numerical equations, which together with (5.12) give
three equation for the three unknowns x, y and . It is often convenient to eliminate at the outset,
which is what (5.11) does. We have proved:
Theorem 56 (Lagranges Theorem). Suppose that f and g are two functions on R2 with continuous
first order partial derivatives. Let B denote the level curve of g given by g(x, y) = 0. Suppose that
g(x, y) 6= 0 at any point along B.
Then if x0 is a maximizer or minimizer of f on B,
#!
"
f (x0 )
=0
det
g(x0 )
5.2.2
(5.13)
Application of Lagranges Theorem
Let us summarize and illustrate our strategy for searching out maximizers and minimizers of a
continuously differentiable function f in a region D bounded by a level curve given by an equation
of the form g(x, y) = 0.
(1) Find all critical points of f in U , the interior of D.
(2) Find all points on B, the boundary of D, at which (5.11) holds.
(3) The combined list of points found in (1) and (2) is a comprehensive list of potential maximizers
and minimizers. Hopefully it is a finite list. In this case, evaluate f at each of them, and see which
produce the largest and smallest values. Case closed.
Example 85 (Finding minimizers and maximizers). Let f (x, y) = x4 + y 4 + 4xy. Let D be the closed
disk of radius 4 centered on the origin. We will now find the maximizers and minimizers of f in D.
We can write the equation for the boundary in the form g(x, y) = 0 by putting
g(x, y) = x2 + y 2 16 .
Part (1): First, we look for the critical points. We have already examined this function in Example 84.
There, we found that f has exactly 3 critical points in all of R2 , and all of them happen to be in the
interior of D. They are
(0, 0)
(1, 1)
and
(1, 1) .
Part (2): Next, we look for solutions of (5.11) and g(x, y) = 0. Since
f (x, y) = 4(x3 + y, y 3 + x)
and
g(x, y) = 2(x, y) ,
(5.14)
"
det
f
g
#!
"
= 8 det
x3 + y
y3 + x
205
#!
= 8(x3 y + y 2 y 3 x x2 ) .
Hence (5.11) gives us the equation x3 y + y 2 y 3 x x2 = 0. Combining this with g(x, y) = 0 we have
the system of equations
x3 y + y 2 y 3 x x2
2
x + y 16
The rest is algebra. The key to solving this system of equations is to notice that x3 y+y 2 y 3 xx2
can be factored:
x3 y + y 2 y 3 x x2 = (x2 y 2 )(xy 1) ,
so that the first equation can be written as (x2 y 2 )(xy1) = 0. Now it is clear that either x2 y 2 = 0,
or else xy 1 = 0.
Suppose that x2 y 2 = 0. Then we can eliminate y from the second equation, obtaining 2x2 = 16,
or x = 2 2. If y 2 = x2 , then y = x, so we get 4 solutions of the system this way:
(2 2, 2 2) .
(5.15)
On the other hand, if xy 1 = 0, y = 1/x, and eliminating y from the second equation gives us
x2 + x2 16 = 0
(5.16)
Multiplying through by x2 , and writing u = x2 , we get u2 16u = 1, so u = 8 63. Since u = x2 ,

p
there are four values of x that solve (5.16), namely 8 63. The corresponding y values are
given by y = 1/x. We obtain the final 4 solutions of (5.11):
(a, 1/a)
with
a = 8 63 .
(5.17)
Part (3): We now round up and interrogate the suspects. There are 11 of them: Three from (5.14),
four from (5.15), and four from (5.17).
206

Now we interrogate the suspects:
f (0, 0)
f (1, 1)
= 2
f (1, 1)

f (2 2, 2 2)

f (2 2, 2 2)

f (2 2, 2 2)
f (2 2, 2 2)
q
q
f ( 8 + 63, 1/ 8 + 63)
q
q
f ( 8 + 63, 1/ 8 + 63)
q
q
f ( 8 63, 1/ 8 63)
q
q
f ( 8 63, 1/ 8 63)
= 2
=
96
96
160
160
258
258
258
258
(5.18)
Evidently, the maximum value of f in D is 258, and the corresponding maximizers are the
four points in (5.17). Also evidently, the minimum value of f in D is 2, and the corresponding
maximizers are the two points (1, 1) and (1, 1). Notice that the maximizers lie on the boundary,
and the minimizers lie in the interior.
Here is a graph showing the boundary curve g(x, y) = 0, which is the circle of radius 4, and
three contour curves of f , namely the contour curves at levels 258, 160, 96, 0, and 1.95. These
are the levels that showed up in our interrogation of the suspects, except that we have used the level
1.95 instead of 2 in the graphplot, since there are just two points, (1, 1) and (1, 1), for which
f (x, y) = 2, and that would not plot well.
Notice that the plot shows 4 points of tangency along the contour curve at altitude 258, and 2
points of tangency along each of the contour curves at altitudes 160 and 96, corresponding exactly to
207
our computations.
Here is a graph showing some contour curves for levels between 160 and 258. As you can see, at
all intermediate levels, the circle cuts across the contour curves, which is what our computations say
should be the case:
We finally close this example with one more observation: We can see from our computations
that the minimum value of f on the boundary is 96.
In our next example, we use Theorem 56 to compute the distance between a point and a parabola.
The idea is to write the equation for the parabola in the form g(x, y) = 0. For instance, if the parabola
is given by y = 2x2 , we can take g(x, y) = 2x2 y.
Suppose for example that we want to find the distance from (1, 3) to this parabola. The square
of the distance from any point (x, y) to (1, 3) is
(x 1)2 + (y 3)2 .
To find the point on the parabola that is closest to (1, 3), we use Theorem 1 to find the point (x0 , y0 )
on the parabola that minimizes (x 1)2 + (y 3)2 this is the point on the parabola that is closest
to (1, 3). Then, by definition, the distance from (1, 3) to the parabola is the distance from (1, 3) to
this closest point.
Before proceeding to the calculations, note that the parabola is closed, but not bounded. Therefore, minima and maxima are not guaranteed to exist. Indeed, the parabola reaches upward for ever
and ever, so there are points on the parabola that are arbitrarily far away from (1, 3). That is, there
is no furthest point.
But is it geometrically clear that there is a closest point. Indeed, since (1, 2) is on the parabola,
and the distance of this point from (1, 3) is 1, we only need to look for the minimum on the part of
the parabola that lies in the closed unit disk centered on (1, 3). This is closed and bounded, so a
minimizer will exist.
This particular problem can be solved using single variables methods: For each x, there is just
208
one point on the parabola, namely (x, 2x2 ), and the squared distance from this point to (1, 3) is
(x 1)2 + (2x2 3)2 = 4x4 11x2 2x + 10 .
Taking the derivative with respect to x, if x is a minimizer of this expression,
16x3 22x 2 = 0 .
This is a cubic equation, and so there are either one or three real roots. In fact, it has three real
roots. Though you could solve for them exactly, the exact answers is very complicated, and involves
cube roots of (108 + 61 7662). However, you can use Newtons method to compute the three roots
numerically, finding
x0 = 1.12419264...
x2 = 0.091465598...
and x3 = 1.21565823...
Evaluating 4x4 5x2 2x + 10 at each of these points, we see that x3 is the minimizer.
We shall now look at this same problem from the point of view of Theorem 56:
Example 86 (Finding the distance to a parabola). Consider the parabola y = 2x2 , and the point
(1, 3). As explained above, to find the point on the parabola that is closest to (1, 3), we minimize
f (x, y) = (x 1)2 + (y 3)2
on the curve g(x, y) = 0 where
g(x, y) = 2x2 y .
Here is a graph showing the parabola, and some of the contour curves of f . As you can see, there
are three places at which the parabola is tangent to the contour curves. Therefore, when we set up
and solve the system consisting of (5.11) and g(x, y) = 0, we will find three solutions, as we found in
our single variable approach.
To apply Theorem 1, we compute f (x, y) = 2(x 1, y 3) and g(x, y) = (4x, 1). Therefore,
(5.11) reduces to
"
0 = det
x1
4x
y3
#!
= 1 + 11x 4xy .
209
Now using 2x2 y = 0, to eliminate y, we are left with
8x3 11x + 1 = 0 ,
which is the same cubic equation that we encountered above. Solving it by Newtons method, we get
the three suspect points:
(x1 , y1 )
(1.12419264... , 2.527618184...)
(x2 , y2 )
(0.0914655981... , 0.01673191127...)
(x3 , y3 )
(1.21565823... , 2.95564990...)
Evaluating f at each of these three points, we find
f (x1 , y1 )
4.73533895...
f (x2 , y2 )
10.0911856...
f (x3 , y3 )
0.048475410...
and clearly (x3 , y3 ) is the closest point. Its distance is the square root of the minimum value found
above, namely 0.220171319..., since f is the squared distance.
The reason we could use single variable methods to solve this problem was that it was easy to
parameterize the parabola. But such explicit parameterizations can difficult to work with in general.
The strength of Theorem 56 is that it allows us to work with implicit descriptions of the boundary.
With more variables, this will be even more useful.
To close this subsection, we work out the minimizers and maximizers for the optimization problem
that we discussed when explaining the tangency condition.
Example 87 (Minimizers and maximizers for f (x, y) = x2 y y 2 x on the unit disk). Let f (x, y) =
x2 y y 2 x, and let D be given by x2 + y 2 1. The boundary is then given by g(x, y) = 0 where
g(x, y) = x2 + y 2 1.
Recall the contour plot we drew before:
210
Notice that the contours of f are symmetric about the line y = x, corresponding to the symmetry
of the function f , as is the circle. Therefore, we can see geometrically that there will be two points
of tangency along the line y = x.
This is very useful information: Knowing one solution to the Lagrange equations simplifies the
algebra required to find all of the others. That is the main point of this example. Let us begin.
Part (1): First, we look for the critical points: f (x, y) = (2xy y 2 , x2 2xy). If (x, y) is a critical
point, then
2xy
y2
2xy
x2 .
(5.19)
This tells us that x2 = y 2 . One solution is x = y = 0. If there is any other solution, neither x nor y
is zero, and we can divide our equations through by a and y, getting
2x
= y
2y
= x.
(5.20)
But then x = 2y = 4x, which is impossible for x 6= 0. Hence (0, 0) is the only critical point.
Part (2): To deal with the boundary, we compute
"
#! "
f
2xy y 2
det
=
g
x2 2xy
2x
2y
Taking the determinant, we deduce 4y 2 x 2y 3 2x3 + 4x2 y = 0. Together with g(x, y) = 0, this gives
us the system of equations
4y 2 x 2y 3 2x3 + 4x2 y
2
x +y 1
0.
211
This may look like a messy system system of non linear equations. Often in these problems, that
is what one gets. Since we can always use Newtons method to solve such a system, that is not so
bad.
However, in this case, one can solve the system using only algebra alone. To do this, notice that
if (x, y) satisfies the first equation, and either y = 0, or y = 0, then both x and y are zero. But then
(x, y) cannot satisfy the second equation. Therefore, if (x, y) is any solution, then neither x = 0 nor
y = 0.
Armed with this information, we can divide the first equation through by y 3 , and defining z = x/y,
the first equation becomes
z 3 + 2z 2 + 2z 1 = 0 .
This is a cubic equation, but we know one solution: We have already seen that there is a point of
tangency with y = x, and hence z = 1.
Indeed, it is easy to see that z = 1 is a root, and knowing one root, it is easy to factor the
cubic:
z 3 + 2z 2 + 2z 1 = (z + 1)(z 2 + 3z 1) .
Factoring the quadratic, we find that the three roots are
z = 1
3+ 5
z=
2
3 5
z=
.
2
and
Since z = x/y, so that y = x/z, we can use these values of z to eliminate y from the equation
2
x + y 2 = 1, getting
x2 (1 + z 2 ) = 1 ,
which mean that
x =
1
1 + z 2
for each of the three values of z found above, and for this same value of z, y = x/z. This gives us
the six points of tangency that are shown in the first plot above.
For z = 1, we get x = 1/ 2, so this value of z gives us the two suspects

(x1 , y1 ) =
For z = (3 +

and
(x2 , y2 ) =
1 1
,
2
2

.
5)/2, we get the two suspects
(x3 , y3 ) =
For z = (3
1 1
,
2
2
!
1+ 5 1 5
,
2 3
2 3
and
(x4 , y4 ) =
!
1 5 1 + 5
,
.
2 3
2 3
5)/2, we get the two suspects
(x5 , y5 ) =
!
1 + 5 1 5
,
2 3
2 3
and
(x6 , y6 ) =
!
1 5 1+ 5
,
.
2 3
2 3
212
Part (3): Now for the interrogation:
f (0, 0)
f (x1 , y1 )
f (x2 , y2 )
f (x3 , y3 )
f (x4 , y4 )
f (x5 , y5 )
f (x6 , y6 )
0
1
= 0.707106781...
2
1
= 0.707106781...
2
r
1 5
= 0.430331483...
3 3
r
1 5
= 0.430331483...
3 3
r
1 5
= 0.430331483...
3 3
r
1 5
= 0.430331483...
3 3
As you can see (x1 , y1 ) is the minimizer, and (x2 , y2 ) is the maximizer.
5.2.3
Optimization for regions with a piecewise smooth boundary
In our next example, we use Theorem 56 to solve an optimization problem in which the boundary is
given in two pieces. This introduces one new feature into the method, as we shall see.
Example 88 (An optimization problem with a two piece boundary). Let D be the region consisting
of all points (x, y) satisfying
x2 y 3 + 2x .
Let f (x, y) = x2 y 3x. We seek to find the minimum and maximum values of f on D, and find all
minimizers and maximizers.
Note that D is a closed, bounded region, and hence minimizers and maximizers in D do exist.
Next, we compute f (x, y) = (2xy 3, x2 ). There are clearly no solutions of f (x, y) = (0, 0).
Hence, the minimizers and maximizers will lie on the boundary.
The first step towards finding them is to plot the two curves defining the boundary of the region
D:
213
The region D lies above the parabola and below the line. Note that the boundary does not have
a tangent direction where the line and parabola meet. Therefore, we have to include these points in
our suspect list: Since there is no tangent at these points, the tangency condition cannot possibly
exclude them.
To find the intersection points, eliminate y from x2 = y and y = 3 + 2x, obtaining
x2 = y = 3 + 2x
and then since x2 2x = 3 has the solutions x = 1, 3, we conclude:
The boundary consists of the points of the parabola y = x2 or on the line y = 3+2x with 1 x 3
in either case. The points where the line meets the parabola, namely ( 1, 1) and (3, 9), must be
included in the suspect list.
We now proceed to check the tangency condition, finding points at which the tangent line to the
contours curves of f are parallel to the tangent lines to the constraint curve. However, as explained
above, the only relevant points are those whose x coordinate lies in the range 1 < x < 3.
Let us first deal with the parabola. We take g(x, y) = x2 y. Then the Lagrange condition
f = g implies that
2x3 = 3 2xy
Combining this with y = x2 , we conclude
4x3 = 3 .
This equation has the unique real solution x = (3/4)1/3 . Hence one possible point to consider is
x1 = ((3/4)1/3 , (3/4)2/3 ) .
Notice the x coordinate is in the relevant range.
214

Now let us consider the linear part of the boundary. This time we take g(x, y) = y 2x 3.
Then the Lagrange condition f = g implies that

2x2 = 2xy 3 .
Substituting in y = 3 + 2x, this becomes 2x2 + 2x = 1, which has the two roots
1 3
x=
.
2
Of these, only ( 3 1)/2 lies in the range of interest, 1 x 3. Using y = 3 + 2x, we find our
next possible point:
x2 = (( 3 1)/2, 3 + 2) .
As explained above, we must included the points where the line and parabola meet, so we round
out the list of suspects with
x3 = ( 1, 1)
and
x3 = (3, 9) .
We now compute
f (x1 )
f (x2 )
f (x3 )
9 1/3 2/3
3 4
2.044260667
16
3
2
3 0.598076212
2
4
f (x4 )
72
We see that x1 is the minimizer, and x4 is the maximizer. Leaving the corners off the suspect
list would have led to a gross underestimation of the maximum.
We next consider another example with a two piece boundary, but of an unbounded region.
Example 89 (Another unbounded region example). Let D be the region consisting of all points
(x, y) satisfying
x2 4 y 4 + x2 .
Let f (x, y) = (x 1)2 + (y 1)2 . We seek to find find the minimum and maximum values of f on
D, and to find all minimizers and maximizers, if any.
The region D is closed but not bounded. Hence we do not have a general guarantee of the existence
of maximizers or minimizers. So let us think before computing. The function f (x, y) is the squared
distance from the point (1, 1). The region D is the region sandwiched between two parabolas that
never intersect. It contains points that are arbitrarily far away from (1, 1). Hence, t f is unbounded
from above on D, and there are no maximizers.
Also since D contains the point (1, 1), at which f is zero, and since f (x, y) > 0 for all other
(x, y), this is the minimizer.
One could solve the Lagrange equations. There would be solutions of f (x, y) = g(x, y), one
on each parabola. A mistake one might make would be to decide the solution of the Lagrange equations
that gives the largest value of f is the maximizer. This is wrong, because as we have seen above, there
is no maximizer.
5.3. THE IMPLICIT FUNCTION THEOREM VIA THE INVERSE FUNCTION THEOREM 215
5.3
The Implicit Function Theorem via the Inverse Function

Theorem
5.3.1
Inverting coordinate tranformations
Recall that in Example 78, we succeeded in parameterizing the Bernouli lemiscate by writing the
equation that defines it in polar coordinates. We shall now prove the Implicit Function Theorem
by constructing a well-behaved system of coordinates in which one of the coordinate functions is a
essentially f itself.
Here is the idea: Let f be a continuously differentiable function on some open set U R2 , and
suppose that at x0 U , f (x0 ) 6= 0. Define the unit vectors
u1 =
1
f (x0 )
kf (x0 )k
and
u2 = u
1
(5.21)
so that {u1 , u2 } is an orthonormal basis for R2 . We next define two functions

u(x) =
1
(f (x) f (x0 ))
kf (x0 )k
v(x) = u2 (x x0 ) .
and
(5.22)
Notice that
u(x) = c
f (x) = d
where
d = ckf (x0 )k + f (x0 )
so that every level set of f is a level set of u, and vice-versa.

Also,
u(x0 ) = v(x0 ) = 0, so if we define a transformation f from U to R2 by
#
" by construction,
u(x)
, we have f (x0 ) = 0.
f (x) :=
v(x)
For example, let f (x, y) = x2 y yx2 and x0 = (1, 1). Then f (x0 ) = (1, 1) so that
1
u1 = (1, 1)
2
1
and u2 = ( 1, 1) .
2
Then, since f (x0 ) = 0,

1
u(x, y) = (x2 y yx2 )
2
and
1
v(x, y) = (x + y 2) .
2
Here is a plot showing contour lines of u(x) and v(x) in the unit square centered on x = x0 :
216

The contour curves of v(x) are straight lines while the contour curves of u(x) have curvature.
However, if we zoom in more, the contour curves of u(x) will look more and more like straight
lines.
Here is a zoomed in contour plot showing contour curves of u(x) and v(x) in the square of
side length 0.1 centered on x0 :
As you can see in these plots, the functions u(x) and v(x) define a system of coordinates in a
neighborhood of x0 . Thus, we can think of the functions u(x) and v(x) as coordinate functions on a
neighborhood of x0 . As we have noted above, every contour curve of u(x) is also a contour curve of
f (x), though generally for a different altitude. Still, each coordinate curve in our coordinate system
is a contour curve of f .
To make effective use of a coordinate system, we not only need to know the coordinate functions
u(x, y) and v(x, y) that specify the new coordinates u and v in terms of x and y, we need the inverse
functions x(u, v) and y(u, v) that express x and y in terms of u and v.
Indeed, when in Example 78 we used the polar coordinate system to parameterize the Bernouli
Lemiscate, we did this by substituting
x(r, ) = r cos
and
y(r, ) = r cos
into f (x, y) = (x2 + y 2 )2 2(x2 y 2 ) = 0 to obtain an equation relating r and .

In the case of the polar coordinate system, it is easy to solve for x and y as a function of the
alternate cooridinates r and . But for other coordinate transformations (u(x, y), v(x, y)) it may not
be possible to explicitly solve for (x(u, v), y(u, v)). In such a case, how can we know that such an
inverse formula even exists?
The key to this is the linear approximation
(u, v) = f (x) f (x0 ) + [Df (x0 )](x x0 ) ,
(5.23)
valid for x close to x0 .

Treating the approximate equality as if it were exact, as in Newtons method, we can now solve
for x, just as in Newtons method. We find, provided that [Df (x0 )] is invertible,
x x0 + [Df (x0 )]1 (u, v) .
(5.24)
Thus under the condition that [Df (x0 )] is invertible, there is a good approximate inverse of our
coordinate transformation, at least in a sufficiently small neighborhood of x0 such that (5.23) is a
good approximation. One might therefore hope that under this condition, there is an exact inverse.
When f is continuously differentiable, this is indeed the case, as we shall show.
First of all, the way we have constructed the functions u(x) and v(x)
that [Df (x0 )]
" guarantees
#
u(x)
is invertible. Indeed, the Jacobian of f , [Df ] is then given by [Df (x)] =
. We compute
v(x)
u(x) =
1
f (x)
kf (x0 )k
so that u(x0 ) = u1 Also,"v(x)

# = u2 for all x.
u1
Therefore [Df (x0 )] =
, and since u1 and u2 are orthonormal,
u2
|det ([Df (x0 )])| = 1 .
Since f , and hence u and v are continuously differentiable, det ([Df (x)]) is a continuous function
of x. By this continuity, there is an r > 0 such that
kx x0 k r
|det ([Df (x0 )])|
1
.
2
In particular, for all x such that kx x0 k r, [Df (x)] is invertible.

We are now ready to state an important theorem, namely the Inverse function Theorem in Rn ,
which gives the n-dimensional analog of what we have been discussing in pictures for n = 2:
Theorem 57 (The Inverse Function Theorem in Rn ). Let f be a continuously differentiable function
on some open set U Rn with values in Rn , and suppose that at x0 U , [Df (x0 )] is invertible.
Then there is an open set V containing x0 and an open set W containing f (x0 ) such that f is a
one-to-one transformation of V onto W , and hence so that the inverse function f 1 from W to V is
well defined. Moreover, f 1 is differentiable at each u W , and [Df (x)] is invertible everywhere on
V , and

1
[Df 1 (u)] = Df (f 1 (u))
.
Before proving the Inverse Function Theorem, we first show how it implies the Implicit Function
Theorem.
5.3.2
From the Inverse Function Theorem to the Implicit Function Theorem
Proof of the Implicit Function Theorem for R2 , Theorem 55: Let f be continuously differentiable on U R2 . Given x0 U , construct u(x) and v(x) as in (5.22), and then form
f (x) = (u(x) , v(x)). Note that by construction, f (x0 ) = 0.
The Inverse Function Theorem provides an open set W containing 0 and an open set V containing x0 such that f is one-to-one from V onto W , and hence invertible, and thus has an inverse
218
1
transfortmation f 1 "
from W
is differentiable everywhere on W . By con# to V , and moreover, f
u1
struction [Df (x0 )] =
where u1 and u2 are given in (5.21). Therefore, by the Implicit Function
u2
Theorem, and the fact that {u1 , u2 } is orthonormal, [Df 1 (x0 )] = [u1 , u2 ].
Since W is open and contains 0, for some a > 0, W contains the open ball {u : kuk < a }.
Since f is continuous and f (x0 ) = 0, there is an r > such that
kx x0 k < r
f (x) {u : kuk < a } .
(5.25)
Now define the curve

x(v) := f 1 (0, v)
a<v <a .
for
This is a differentiable curve since f 1 is differentiable. Also, x(0) = f 1 (0) = x0 since f (x0 ) = 0.
By the chain rule, and what we have noted just above,
x0 (0) = [Df 1 ( 0)]e2 = [u1 , u2 ]e2 = u2 .
In particular, x0 (0) 6= 0.
Next, for all a < v < a,
(u(x(v)), v(x(v))) = f (x(v)) = f (f 1 (0, v)) = (0, v) .
In particular, u(x(v)) = 0 for all a < v < a.
But by the definition (5.22) of u(x),
u(x) = 0
f (x) = f (x0 ) .
Also, by our construction of f ,

u(x) = 0
f (x) = (0, v)
for some
a<v <a .
for some
a<v <a .
Thus, for x such that kx x0 k < r,

f (x) = f (x0 )
x = x(v)
This means that every solution of f (x) = f (x0 ) such that kx x0 k < r lies on the curve x(v)
for some a < v < a, and moreover, since f is one-to-one on W , there is only one such v.
5.3.3
Proof of the Inverse Function Theorem.
We turn to the proof of the Inverse Function Theorem.

We first show that f is one-to-one on a neighborhood of x0 , assuming for now that [Df (x0 )]
has orthonormal rows, like the transformation f that we constructed to prove the Implicit Function
Theorem. Before stating the lemma, we recall some notation form Chapter 3: For any r > 0, Br (x0 )
denotes the open ball of radius r centered on x0 . That is,
Br (x0 ) = {x : kx x0 k < r } .
Lemma 12 (The one-to-one property). Let f be a continuously differentiable function on U Rn
with values in Rn . Let x0 U , and suppose that [Df (x0 )] is invertible. Then there is an r > 0 so
that
x, y Br (x0 )
kf (x) f (y)k
1
kx yk
2k[Df (x0 )]1 kF
(5.26)
and
x Br (x0 )
[Df (x)]
is invertible .
(5.27)
In particular, whenever x and y belong to Br (x0 ) and f (x) = f (y), then x = y.

Proof: By the Chain Rule and the Fundamental Theorem of Calculus, if we define x(t) := x+t(yx),
so that x(0) = x and x(1) = y, and x0 (t) = y x,
Z
f (y) f (x) =
[Df (x(t))](y x)dt .

0
For x and y close to x0 , we have [Df (x + t(y x)] [Df (x0 )] since f is continuously differentiable.
This approximation gives us
1
Z
f (y) f (x)
[Df (x0 ](y x)dt = [Df (x0 )](y x) ,
(5.28)
where we have used the fact that in this approximation, the integrand is constant. Thus, we would
have y x [Df (x0 )]1 (f (y) f (x)). Applying Theorem 44, we would have

ky xk / [Df (x0 )]1 F kf (x) f (y)k .
Now we simply control the size of the errors made in this approximation, showing they cost us
no more than a factor of 2. An exact form of (5.28) is
Z
f (y) f (x)
([Df (x0 )](y x) + ([Df (x0 )](x(t)) [Df (x0 )](y x))dt
=
0
Z
[Df (x0 )](y x) +
([Df (x0 )](x(t)) [Df (x0 )](y x))dt .
(5.29)
Hence, multiplying through by [Df (x0 )]1 and using the triangle inequality and Theorem 44,
kx yk
k[Df (x0 )]1 kF kf (y) f (x)k

Z 1
+ ky xk
k[Df (x0 )]1 kF k[Df (x0 )](x(t)) [Df (x0 )]kdt
Since f is continuously differentiable, the real valued function (x)

(x) := k[Df (x0 ](x(t)) [Df (x0 ]kF
is continuous, and (0) = 0. Hence there is an r > 0 such that
kx x0 k < r
(x) <
1
.
2k[Df (x0 )]1 kF
(5.30)
220
Notice tnat when x and y are in Br (x0 ), then x(t) Br (x0 ) for all 0 t 1 (since balls are convex).
Then, for x, y Br (x0 ), the integrand in (5.30) is no greater than 1/2 for any t, and hence for
x, y Br (x0 ), (5.30) gives
1
kx yk k[Df (x0 )]1 kF kf (y) f (x)k + kx yk .
2
1
kx yk from both sides, this gives us (5.26). Finally, since f is continuously differen2
tiable, and since the set of invertible matrices is open, by further decreasing r we may arrange that
Cancelling
(5.27) is valid.
Lemma 13 (The onto property). Let f be a continuously differentiable function on U Rn with

values in Rn . Let x0 U , and suppose that [Df (x0 )] is invertible. Let r > 0 be such that (5.26) and
(5.27) are valid, and define
r0 =
1
.
2k[Df (x0 )]1 kF
Then if y belongs to the ball Br0 /2 (y0 ), where y0 := f (x0 ), there exists x Br (x0 ) such that f (x) = y.
Proof: Fix any y Br0 /2 (y0 ). We must find a x Br (x0 ) such that f (x) = y. Define the real
valued function
g(x) := kf (x) yk2 .
If there is a x Br (x0 ) such that f (x) = y, then x is a minimizer of g on Br (x0 ). Let C be the
closed unit ball of radius r about x0 in Rn . Since C is compact, and since g is continuous, there is a
minimizer of g on C. We claim that the minimizer cannot belong to the boundary. To see this, note
that if x belong to the boundary, so that kx x0 k = r, (5.26) gives us
kf (x) y0 k = kf (x) f (x0 )k >
r
= r0 .
2k[Df (x0 )]1 kF
By the triangle inequality,

kf (x) yk kf (x) y0 k ky y0 k .
Thus for y Br0 /2 (y0 ), kf (x) yk r0 /2 so that
g(x) = kf (x) yk2
r02
.
4
On the other hand, when y Br0 /2 (y0 ),

g(x0 ) = kf (x0 ) yk2 = ky0 yk2 <
r02
.
4
This shows that g(x0 ) < g(x) whenever x belongs to the boundary of C, and hence no such point can
be a minimizer. Hence, there exists a minimizer x of g in the interior of C. Since g is differentiable,
it must be the case that at such a minimizer x,
g(x) = 0
and by the chain rule
g(x) = 2[Df (x)](f (x) y) .
5.4. THE GENERAL IMPLICIT FUNCTION THEOREM
221
By (5.27), [Df (x)] is invertible, and hence f (x) y = 0; that is, f (x) = y.
Proof of the Inverse Function Theorem, Theorem 57: Let x0 , y0 , r and r0 be as in Lemma 13.
By Lemmas 12 and 13, for each y Br0 /2 (y0 ), there is a unique x Br (x0 ) such that f (x) = y.
Hence on f 1 is well defined on Br0 /2 (y0 ). It remains to show that f 1 is differentiable at y0 , and
that the derivative there is [Df (x0 )]1 .
To do this, fix y Br0 /2 (y0 ), and let x := f 1 (y). We must show that for all > 0, there exists
a > 0 such that
ky y0 k <
kf 1 (y) f 1 (y0 ) [Df (x0 )]1 (y y0 )k < ky y0 k .
(5.31)
However,
f 1 (y) f 1 (y0 ) [Df (x0 )]1 (y y0 )
=
x x0 [Df (x0 )]1 (f (x) f (x0 ))
[Df (x0 )]1 (f (x) f (x0 ) [Df (x0 )](x x0 )) .
(5.32)
Hence by Theorem 44,

kf 1 (y) f 1 (y0 ) [Df (x0 )]1 (y y0 )k
k[Df (x0 )]1 kF kf (x) f (x0 ) [Df (x0 )](x x0 )k . (5.33)
Since f is differentiable at x0 , for any b
> 0, there exists a bb > 0 such that
kx x0 k < bb
kf (x) f (x0 ) [Df (x0 )](x x0 )k b

kx x0 k.
(5.34)
By (5.26),
kx x0 k
1
ky y0 k ,
2k[Df (x0 )]1 kF
and hence
ky y0 k <
bb
2k[Df (x0 )]1 kF
kx x0 k < bb .
(5.35)
Combining (5.33), (5.34) and (5.35), we conclude

ky y0 k <
bb
2k[Df (x0 )]1 kF
kf 1 (y) f 1 (y0 ) [Df (x0 )]1 (y y0 )k < k[Df (x0 )]1 kF b

.

bb
, and :=
, we see that (5.31) is valid. Thus,
1
k[Df (x0 )] kF
2k[Df (x0 )]1 kF
is differentiable at y0 .
Now choosing b
:=
f 1
5.4
The general Implicit Function Theorem
We have proved the Inverse Function Theorem for functions f from Rn to Rn . So far, we have only
discussed the Implicit Function Theorem for functions on R2 . As we now show, the method we
222
have used to deduce the Implicit Function Theorem for functions on R2 from the Inverse Function
Theorem readily generalizes to yield a general Implicit Function Theorem in all dimensions.
For the Implicit Function Theorem on R2 , recall that the basic idea is this: Given a continuously
differentiable function f (x, y) such that f (x0 , y0 ) 6= 0, construct another continuously differentiable
function g(x, y) such that h(x, y) := (f (x, y), g(x, y)) has an invertible Jacobian at (x0 , y0 ). Then
apply the Inverse Function Theorem to h.
Now consider a function f defined on an open set U in Rn with values in Rk where f (x) =
(f1 (x), . . . , fk (x)). If f is continuously differentiable in U , then for x U , the k n matrix [Df (x)]
is given by
f1 (x)
..
.
[Df (x)] =
.
fk (x)
In analogy with what we did in R2 , we would like to think of {f1 , . . . , fk } as the first k functions
in a system of n coordinate functions {f1 , . . . , fn } on Rn in some open set U containing some point
x0 . Thus, we must flesh out the given functions {f1 , . . . , fk } with another n k continuously
differentiable functions {fk+1 , . . . , fn } in such a way that if we define
h(x) := (f1 (x), . . . , fk (x), fk+1 (x) . . . , fn (x)) ,
(5.36)
the Jacobian of h at x0 is invertible; i.e., [Dh (x0 )] is invertible.

Whatever choice we make for {fk+1 , . . . , fn }, when h is defined by (5.36), [Dh (x0 )] is invertible
if and only if
{f1 (x0 ), . . . , fk (x0 ), fk+1 (x0 ), . . . , fn (x0 )}
is linearly independent.
Of course, whenever {f1 (x0 ), . . . , fk (x0 ), fk+1 (x0 ), . . . , fn (x0 )} is linearly independent,
so is {f1 (x0 ), . . . , fk (x0 )}.
On the other hand, when {f1 (x0 ), . . . , fk (x0 )} is linearly independent, we can alway keep
adding vectors to it until we achieve a maximal linearly independent set of n vectors {f1 (x0 ), . . . , fk (x0 ), uk+1 , .
We even have several constructive methods for doing this.
However, we do it, if we define fj (x) by
fj (x) := uj (x x0 )
for
j = k + 1, . . . , n ,
and then use these functions to define h(x), we shall have
f1 (x0 )
..
f (x )
k
0
[Dh (x0 )] =
uk+1
..
un
(5.37)
5.4. THE GENERAL IMPLICIT FUNCTION THEOREM
223
By construction, the rows are linearly independent, so [Dh (x0 )] is invertible. This enables us to
adapt the strategy we have used to prove the Implicit Function Theorem in R2 to prove a much more
general result. As noted above, the key condition will be that {f1 (x0 ), . . . , fk (x0 )} is linearly
independent, which is that same as [Df (x0 )] having rank k.
Theorem 58 (Implicit Function Theorem in Rn ). Let f be an Rk valued function defined on an open
set U Rn , and suppose that f is continuously differentiable at each point x0 U . Then, in case
[Df (x0 )] has rank k, there is an r > 0 and a differentiable function g defined on the open ball of
radius r in Rnk so that
(1) For all (uk+1 , . . . , un ) with
Pn
j=k+1
u2j < r2 , g(uk+1 , . . . , un ) = f (x0 ),
and
(2) There is an s > 0 such that every x satisfying f (x) = f (x0 ) and kx x0 k < s is of the form
x = g(uk+1 , . . . , un )
for exactly one (uk+1 , . . . , un ) such that
Pn
j=k+1
u2j < r2 .
Proof: Suppose that [Df (x0 )] has rank k. Then we can select vectors {uk+1 , . . . , un } so that
{f1 (x0 ), . . . , fk (x0 ), uk+1 , . . . , un } is linearly independent. We then define fj (x), j = k + 1, . . . , n
by (5.37) and then h by (5.36). Then by what we have explained above, and by the Inverse Function
Theorem, there is an open set V Rn containing x0 and an open set W Rn containing h(x0 )
such that h is a one-to-one transformation from V onto W By the construction of {fk+1 , . . . , fn },
fj (x0 ) = 0 for j = k + 1, . . . , n. Thus,
h(x0 ) = (f1 (x0 ), . . . , fk (x0 ), 0, . . . , 0) W .
Since for all y W , h(h1 (y)) = y, and since for each hh = fj for each j = 1, . . . k,
fj (h1 (y)) = hj (h1 (y)) = yj
for each
j = 1, ...k .
(5.38)
Since W is open, there is an r > 0 so that

u2k+1 + u2n < r
(f1 (x0 ), . . . , fk (x0 ), uk+1 , . . . , un ) W .
f denote the set of vectors (uk+1 , . . . , un ) such that u2 + u2 < r. Define the function
Let W
n
k+1
n
f
g from W to R by

g(uk+1 , . . . , un ) := h1 f1 (x0 ), . . . , fk (x0 ), uk+1 , . . . , un ,
we can rephrase (5.38) as saying

f g(uk+1 , . . . , un ) = f (x0 ) .
f , g(uk+1 , . . . , un ) varies over the set of solutions of the
Thus, as (uk+1 , . . . , un ) varies over W
equation f (x) = f (x0 ).
224
The final point to show is that all solutions with x sufficiently close to x0 are of the form
f.
x = g(uk+1 , . . . , un ) for some (uk+1 , . . . , un ) W
It suffices to show that for some s > 0, whenever f (x) = f (x0 )
kx = x0 k < s
f,
(hk+1 (x), . . . , hn (x)) W
(5.39)
since in this case, by the inverse property and the definition of g,

x = h1 (h(x))
= h1 (f1 (x0 ), . . . , fk (x0 ), hk+1 (x), . . . , hn (x))

= g(hk+1 (x), . . . , hn (x)) .
To find an s for which (5.39) is valid, we use the continuity of h, remembering that h is even
differentiable. Hence there is an s > 0 so that
kx = x0 k < s
kh(x) h(x0 )k < r .
Since for j > k, hj (x0 ) = 0,

kh(x) h(x0 )k < r
v
u X
u n
t
h2j (x) < r ,
j=k+1
which gives us (5.39).
5.5
Implicit curves and surfaces in three or more dimensions
Let f be a continuously differentiable function on R3 with values in R. For example,

f (x, y, z) = x2 + y 2 + z 2 .
In this case, the level set of f consisting of all the solutions of the equation f (x, y, z) = 1 is the unit
sphere in R3 . The equation f (x, y, z) = 1 is the implicit definition of this surface.
The unit sphere is a particularly nice surface, so we can also explicitly parameterize it. Here is
one way: Define
x(s, t) = ( cos s sin t , sin s sin t , cos t) ,
where 0 s < 2 and 0 t < .
Likewise, let f be a continuously differentiable function on R3 with values in R2 . For example,
"
#
x2 + y 2 + z 2
f (x, y, z) =
.
z
In this case, the level set of f consisting of all the solutions of the equation f (x, y, z) = (1, 0) is the
intersection of unit sphere in R3 with the plane z = 0. This is nothing other than the unit circle in
x, y plane, which can be explicitly parameterized by
x(t) = ( sin(t), cos(t), 0) .
5.5. IMPLICIT CURVES AND SURFACES IN THREE OR MORE DIMENSIONS
225
The Implicit Fucntion Theorem says that whenever f is differentiable from R3 to R, and
f (x0 ) 6= 0, the equation f (x) = f (x0 ) defines a differentiable parameterized surface in R3 passing
through x0 , and including all of the solutions of f (x) = f (x0 ) in some neighborhood of x0 .
Given a system of two such equations f (x) = f (x0 ) and g(x) = g(x0 ), with f (x0 ) 6= 0 and
g(x0 ) 6= 0 each will determine a surface passing through x0 . Then provided that
f (x0 ) g(x0 ) 6= 0 ,
so that f (x) := (f (x), g(x)) has a Jacobian [Df (x0 )] of rank 2 at x0 , the Implicit Function Theorem
says that their intersection will be a differentiable parameterized curve passing through x0 , and the
points on this curve will be the solutions to the system.
That is, just as with lines and planes in R3 , where a (x x0 ) = 0 specifies a plane, but it takes
a system of two such equations to specify a line, one equation f (x) = f (x0 ) specifies a surface in a
neighborhood of x0 , provided f (x0 ) 6= 0, however it takes two such equations f (x) = f (x0 ) and
g(x) = g(x0 ) to specify a curve passing through x0 .
If the surfaces in question are more complicated than planes or spheres and such, it might not be
possible to find explicit parameterizations, but the Implicit Function Theorem for R3 at least assures
us that such parameterizations exist.
5.5.1
Constrained optimization in three variables
There are several types of constrained optimization problems in three variables. Let us begin with
one of the simplest.
Let f and g be a continuously differentiable functions defined on R3 , and define the sets D, U
and B by
D
:= {(x, y, z) : g(x, y, z) 0 }
:= {(x, y, z) : g(x, y, z) < 0 }
:= {(x, y, z) : g(x, y, z) = 0 } .
(5.40)
Then U is the interior of D and B is its boundary.

Let us try to determine the minimizers and maximizers of f on D, if any. Just as before, the
only points in U that can possibly be minimizers or maximizers are the critical point of f in U .
Next, we consider the boundary. Let x0 be any point in B, and assume that g(x0 ) 6= 0. Let
x(s, t) be a parameterization of the surface specified g(x) = g(x0 ) = 0 in a neighborhood of x0 ; the
Implicit Function Theorem assures us that such a parameterization exists. We then compute

d
f (x(s, 0))
= f (x0 )
x(0, 0) ,
ds
s
s=0
and
f (x(0, t))
= f (x0 ) x(0, 0) .
dt
t
t=0
Unless both of these are zero, one can move along the surface given by g(x) = 0 and strictly increase
or decrease the value of f .
226

Therefore, for x0 to even possibly be a minimizer or maximizer, we must have that

d
= f (x0 )
f (x(s, 0))
x(0, 0) = 0 ,
ds
s
s=0
and
d
= f (x0 ) x(0, 0) = 0 .
f (x(0, t))
dt
t
t=0
Thus, f (x0 ) must be orthogonal to both

x(0, 0) and to
x(0, 0). Therefore, f (x0 ) must
s
t
be a multiple of
x(0, 0) x(0, 0).
s
t
Then since g(x(s, t) = 0 for all s, t, the same analysis shows that g(x0 ) is a multiple of
x(0, 0) x(0, 0). Thus, since by assumption g(x0 ) 6= 0, it follows that for some ,
s
t
f (x0 ) = g(x0 ) .
This in turn is equivalent to f (x0 ) g(x0 ) = 0. Notice also that this equation is satisfied
also in case g(x0 ) = 0. Therefore we have proved:
Theorem 59 (Lagranges equation in R3 for a single constraint). Let f and g be continuously
differentiable functions defined on R3 , and define the sets D, U and B by (5.40). Then if x0 is a
minmizer or a maximizer of f on B, x0 satisfies the system of equations
f (x0 ) g(x0 )
g(x0 )
= 0
=
Notice the first equation in the system, being a vector equation in R3 , is really three scalar
equations. Thus, we have four equations in three unknowns, x, y and z. However, this is on account
of harmless redundancy, just like the redundancy we have when we write the equation of a line in R3
in the form a (x x0 ) = 0.
Example 90 (One constraint in three variables). Let f (x, y, z) = xyz, and let g(x, y, z) = x2 + y 2 +
z 2 1. Let us find the minimizers and maximizers of f in the set D given by g(x) 0, which is the
closed unit ball.
We first compute
f (x, y, z) = (yz, xz, zy) .
Clearly 0 is one critical point. Next, suppose (x, y, z) is any critical point with x 6= 0. Then dividing
(yz, xz, zy) = (0, 0, 0) through by x, we find
(yz/x, z, y) = (0, 0, 0) .
Thus, y = 0 and z = 0. Hence all of the points
(x, 0, 0)
with
1x1
are critical points of f in D. By symmetry in x, y and z, so are the points (0, y, 0) with 1 y 1
and (0, 0, z) with 1 z 1. Thus we have infinite many critical points. However, f takes on the
same value, namely 0, at each of them, so this is not so bad.
5.5. IMPLICIT CURVES AND SURFACES IN THREE OR MORE DIMENSIONS
227
Next, to look for possible minimizers and maximizers on the boundary B, we compute
g(x, y, z) = 2(x, y, z) ,
and then
f (x) g(x) = 2((y 2 z 2 )x , (z 2 x2 )y , (x2 y 2 )z) .
Thus if x = (x, y, z) is on the boundary and is a minimizer or a maximizer, then
(y 2 z 2 )x
(z x )y
(x2 y 2 )z
x2 + y 2 + z 2
1.
From the first equation we see that either x = 0 or y 2 = z 2 . Suppose x = 0. Then from the
second and thrid eqautions, we have that either y = 0 or z = 0. Suppose y = 0. Then from the last
equation, we have z 2 = 1 so z = 1. Thus we find the solution (0, 0, 1). If instead we took z = 0,
we would find the solutions (0, 1, 0).
On the other hand, if y 2 = z 2 , either y = z = 0, in which case the fourth equation tells us
x = 1, giving us the solutions ( 1, 0, 0), or else y 2 = z 2 > 0, and then the second equation tells us
x2 = z 2 as well. Thus, using the fourth equation,
x2 = y 2 = z 2 =
1
.
3
This gives us 8 more solutions, namely

( 31/2 , 31/2 , 31/2 ) .
By symmetry, we have found all of the solutions, so now let us evaluate f at each of them. Only
the last 8 give non-zero values.
f ( 31/2 , 31/2 , 31/2 ) = 33/2
with the plus sign n the right if there are an even number of minus signs on the left, and the minus
sign on the right otherwise. This gives all of the minimizers and maximizers.
Next, let us consider the problem of finding minmizers and maximizers along an implicitly defined
curve in R3 given by the system of equations
g(x) = 0
where
"
g(x) =
g1 (x)
g2 (x)
#
.
Let us assume that g1 (x0 ) g2 (x0 ) 6= 0 so that the Implicit Function Theorem does ensure that
the solution of g(x) = 0 near x0 is given by a differentiable curve x(t) passing through x0 at t = 0.
228

Then since both g1 (x(t)) and g2 (x(t)) are constant, we have

d
d
0

= g1 (x0 ) x (0) = 0 = g2 (x(t))
= g2 (x0 ) x0 (0) .
g1 (x(t))
dt
dt
t=0
t=0
Thus, x0 (0), which is not zero, must be a nonzero multiple of g1 (x0 ) g2 (x0 ).
On the other hand, if x0 is a maximizer of a minimizer of f on the level set given by g(x) = 0,
it must be the case that

d
= f (x0 ) x0 (0) = 0 .
f (x(t))
dt
t=0
Thus, f (x0 ) must be orthogonal to x0 (0) which is in turn a non-zero multiple of g1 (x0 ) g2 (x0 ).
There are two ways we can express this. One is the traditional Lagrange multiplier formulation:
There exist numbers and such that
f (x0 ) = g1 (x0 ) + g2 (x0 ) .
The other, which eliminates and from the outset, is simply to write
f (x0 ) g1 (x0 ) g2 (x0 ) = 0 .
The latter is usually more convenient in three dimensions, however the former does not involve
the cross product, and therefore may be readily generalized to higher dimentions. We have proved:
Theorem 60 (Laganges equation in R3 for a two constraints). Let f and g = (g1 , g2 ) be continuously
differentiable functions defined on R3 , with values in R and R2 respectively. Then if x0 is a minmizer
or a maximizer of f on the solution set of g(x) = 0, x0 satisfies the system of equations
f (x0 ) g1 (x0 ) g2 (x0 )
g(x0 )
= 0
Example 91 (Two constraints in three variables). Let f (x, y, z) := xyz, and let g1 (x, y, z) :=
x2 + y 2 + z 2 1 and g2 (x, y, z) = 2x + 2y z. Let us find the minimizers and maximizers of f on
the curve in R3 given implicitly by g(x) = 0.
Writing out g(x) = 0 explicitly as a system of equations, it becomes
x2 + y 2 + z 2
2x + 2y z
Thus, the solution set is the intersection of the unit sphere and a plane through the origina, and
it is therefore a great circle (geodesic) on the unit sphere. We could of course easily parameterize it.
But let us instead solve the problem by means of Theorem 60.
We compute
g1 (x) g2 (x) = 2(x, y, z) (2, 2, 1) = 2((y 2z , 2z + x , 2x 2y) .
Next, we compute f (x) = (yz, xz, zy), and so
f (x) g1 (x) g2 (x) = 2(yz(y + 2z) + xz(2z + x) + xy(2x 2y)) .
5.6. EXERCISES
229
Thus, after some grouping, we have the system of equations

z(x2 y 2 ) + 2z 2 (x y) + 2xy(x y)
x2 + y 2 + z 2
2x + 2y z
Substituting z = 2(x+y) from the third equation into the first equation, the first equation becomes
10(x + y)2 (x y) = 2xy(x y) .
Thus, either x = y, or
10(x + y)2 2xy = 0 .
The latter is a quadratic equation, and its only solution is x = y = 0, which is a special case of x = y.
Thus, we must have x = y.
Now the last two equations in our system simplify to z = 4x and 2x2 + z 2 = 1. Eliminating z,
18x2 = 1 so we have
2
y=x=
3 2
and then from z = 4x, we find our candidates
2
(1, 1, 4) .
3 2
We then evaluate to find
f

2
1
(1, 1, 4) = 33/2 ,
2
3 2
2
2
and hence (1, 1, 4) is the maximizer and (1, 1, 4) is the minimizer.
3 2
3 2
Comparing with Example 90, we see that imposing the additional constrain raised the minimum
value and lowered the maximum value, as one would expect.
5.6
Exercises
5.1: Let f : R2 R be given by

f (x, y) = x3 y + 2y 3y 2 x .
f is horizontal.
(b) Could the following be a contour plot of f ? Explain your answer.
230

3
2
y
1
x
1
2
3

f (x, y) = x2 y + yx xy 2 .
(a) Compute the gradient of f , and find all critical points of f .

(b) Find a parameterization of the tangent line to the level curve of f through the point (1, 1).
(c) Could either of the following be a contour plot of f ? If so, which could, and which could not?
Explain your answer.

f (x, y) = x3 + y 3 + 3xy .
f is horizontal.
(b) Find the equation of the tangent line to the level curve of f passing through the point (1, 1).
(c) Could the following be a contour plot of f ? Explain your answer.
5.6. EXERCISES
231
xy
.
(1 + x2 + y 2 )2
(a) Find all of the critical points of f , and find the value of f at each of the critical points.
5.4: Let f (x, y) =
(b) One of the following is a contour plot for f . Which one is it? Explain your answer to receive
credit.
5.5: Let f (x, y) = xy 2 xy.

(a) Find all of points at which the tangent plane to the graph of f is horizontal.
(b) Find the equation of the tangent line to the contour curve of f through the point (3/2, 1/3).
(c) Could the following be a contour plot for f ? Explain your answer to receive credit.
232
5.6: Let f (x, y) = (x + y)4 + (x y)2 . Find the minimum and maximum values of f on the unit
circle x2 + y 2 = 1, and all of the places on the circle at which f takes on these values.
5.7: Let f (x, y) = xy. Let D denote the region in the plane consisting of all of the points (x, y) such
that x2 + 4y 2 6. Find the minimum and maximum values of f in this region. Also, find all of the
minimizers and maximizers in this region.
xy
. Find the minimum and maximum values of f in the set where
5.8: Let f (x, y) =
(1 + x2 + y 2 )2
|x| + |y| 1 .
Also, find all of the minimizers and maximizers.
5.9: Let f (x, y) = xy. Find the minimum and maximum values of f the set D R2 given by
(x2 + y 2 )2 x2 y 2 ,
and all of the points in D at which f takes on these values. Note that D is the region inside the
Bernoulli lemiscate.
5.10 Let S be closed upper hemisphere of the unit sphere in R3 . Let f (x, y, z) = xyz. Find the
minimum and maximum values of f on S, and all of the points at which f takes on these values.
Explain how you are taking into account both of the constraints x2 + y 2 + z 2 = 1 and z 0.
5.11: Let f (x, y) = x2 + y 2 xy. Find the minimum and maximum values of f the closed unit disk
D R2 . That is, D is given by
x2 + y 2 1 .
5.12: Let f (x, y) = x2 + 2y. Find the minimum and maximum values of f in the set D R2 that
lies below the parabola y = 2x x2 and inside the circle (x 1)2 + y 2 = 1.
5.13: Let f (x, y) = xy + 2x 2y. Find the minimum and maximum values of f on the region D R2
that lies below the parabola y = 1 x2 and above the x-axis, y = 0.
5.14: Let D be the region consisting of all points (x, y) satisfying
x2 y 3 + 2x .
Let f (x, y) = x2 y 3x. We seek to find the minimum and maximum values of f on D, and find all
minimizers and maximizers.
5.15: Let f (x, y) = x2 + y 2 2xy. Find the minimum and maximum values of f the region D defined
by
x2
y2.
2
Also, find all points in D at which f takes on these values; i.e., find all of the minimizers and
maximizers.
5.16: Find the points on the ellipsoid given by x2 + y 2 + xy = 1 that minimize and maximize the
distance to the line y = 3 2x. (Draw a picture, and think about the geometric idea behind the idea
of Lagrange multipliers before plunging into computation.)
5.6. EXERCISES
233
5.17: Find the maximum and minimum values of f (x, y, z) = 3x + y z on the set C of points in
R3 satisfying
x+y+z
2
x +y
1.
Notice that C is the intersection of a plane and a cylinder about the z-axis.
5.18: Find the points on the sphere x2 + y 2 + z 2 = 1 that are closest and farthest from the point
(1, 2, 3).
5.19: Find the point on the paraboloid z = x2 + y 2 that is closest and farthest from the point
(1, 3, 4).
234
Chapter 6
CURVATURE AND QUADRATIC

APPROXIMATION
6.1
6.1.1
The best quadratic approximation

Higher order directional derivatives and repeated partial differentiation
Let f : Rn R. Given x0 and v Rn , we then define, as usual, the slice function g : R R

by g(t) = f (x0 + tv). The higher order derivatives of g at t = 0, if they exists, are the higher order
directional derivatives of f at x0 in the direction v.
By the chain rule
g 0 (t) = v f (x0 + tv) .
(6.1)
Taking higher derivatives will tell us how the graph of z = f (x) curves away from its tangent
plane at a given point x0 . Does it curve up, or curve down, or does it curve up in some directions
and down in others? In many applications of multivariable calculus, it is important to be able to
answer these questions. And as we know, understanding curvature involves second derivatives. There
are two issues before us: We need to develop an efficient means of computing higher order directional
derivatives, and then, once we know how to compute them, we need to understand what they are
telling us about the geometry of the graph of z = f (x). In the rest of this subsection, we focus on
the issue of how to compute.
To take a second derivative, it helps to write the dot product in (6.2) as an explicit sum:
0
g (t) =
n
X
i=1
vi
f (x0 + tv) .
xi
(6.2)
Then since the vi do not depend on t, and since the derivative of a sum is the sum of the derivatives,

235
236
CHAPTER 6. CURVATURE AND QUADRATIC APPROXIMATION
we can apply the chain rule once more to compute

n
X
d
g 00 (t) =
vi
f (x0 + tv)
dt xi
i=1

n
X
=
f (x0 + tv)
vi v
xi
i=1
n
X
vi vj
i,j=1

f (x0 + tv) .
xj xi
(6.3)

f denotes the function you get by first
xj xi
taking the xi partial derivative of f , and then taking the xj partial derivative of that. The following
To be clear about the notation we are using,
more compact notation is often used to denote the same thing:

2
f (x1 , . . . , xn ) ,
xj xi
and in case j = i, it is common to write
2
f (x1 , . . . , xn ) .
x2i
We shall use these notations interchangeably, which is the usual practice.
Example 92. Let f (x1 , x2 ) = x31 + x32 3x1 x2 . Then
f (x1 , x2 ) = 3x21 3x2 .

x1
and so
2
f (x1 , x2 ) = 6x1
x21
2
f (x1 , x2 ) = 3 .
x2 x1
and
Likewise,
f (x1 , x2 ) = 3x22 3x1 .

x2
and so
2
f (x1 , x2 ) = 6x2
x22
2
f (x1 , x2 ) = 3 .
x1 x2
and
Notice that computing second partial derivatives is only a matter of computing one variable
derivatives with respect to various pairs of variables.
The formula

n
X

2
d2

f
(x
+
tv)
=
v
v
f (x0 )
0
i
j

dt2
xj xi
t=0
i,j=1
(6.4)
can be written in matrix notation, and this turns out to much more useful than one might first guess.
Definition 63 (Hessian matrix). Let f be a function defined on an open set U Rn with values in
R such that all of the second order partial derivatives of f exist and are continuous in U . Then at
any x U , the Hessian matrix of f at x is the n n matrix [Hessf (x)] whose i, jth entry is
[Hessf (x)]i,j :=
2
f (x) .
xj xi
6.1. THE BEST QUADRATIC APPROXIMATION
237
The point of the definition is that

([Hessf (x)]v)i =
n
n
X
X
[Hessf (x)]i,j vj =
vj
j=1
j=1
2
f (x)
xj xi
and so
v [Hessf (x)]v =
n
X
vi vj
i,j=1
2
f (x0 ) .
xj xi
Therefore, we can rewrite our formula (6.4) as

d2
= v [Hessf (x)]v .
f (x0 + tv)
2
dt
t=0
(6.5)
Example 93. Let f (x1 , x2 ) = x31 +x32 3x1 x2 as in Example 92. Let us compute the second order

d2
directional derivative 2 f (x0 + tv)
for
dt
t=0
x0 := ( 1, 1)
and
v = (u, v) .
That is, we take v1 = u and v2 := v.

"
By our computations in Example 92, [Hessf (x)] =
"
[Hessf (1, 1)] = 3
6x1
6x2
. Therefore,
#
.
Now we compute
"
v [Hessf (1, 1)]v
3(u, v)
3(u, v) (2u + v, u + 2v)
6(u2 + uv + v 2 ) .
(u, v)
Since we have computed the second directional derivative for an arbitrary direction (u, v), we can
ask which directions give the minimal curvature, and which give the maximal curvature.
That is, let us plug in
v = (u, v) = ( cos , sin ) .
Then (6.5) and (6.6) yield

d2

f
(x
+
t(
cos(),
sin()))
0

2
dt
t=0
= 6 cos2 () 6 sin() cos() 6 sin2 ()

= 6 3 sin(2) .
(6.6)
This is always negative, and is maximized at = /4 and 5/4 where sin(2) = 1, and minimized at
= 3/4 and 7/4 where sin(2) = 1.
238

What we have done so far, we can repeat. One more repetition will suffice for our purposes. Let
us compute g 000 (t) starting from (6.3). Since the derivative of a sum is the sum of the derivatives,
and since the components of v are independent of t.
000
g (t) =
n
X
j,k=1
d
vj vk
dt

f (x0 + tv) .
xj xk
Applying the chain rule once more, we have for each j and k,

d

f (x0 + tv)
= v
f (x0 + tv)
dt xj xk
xj xk
n
X

=
vi
f (x0 + tv) .
x
i xj xk
i=1
Combining the last two calculations, we deduce
000
g (t) =
n
X

vi vj vk
i,j,k=1

f
xi xj xk

(x0 + tv) .
(6.7)
Now, how large is |g 000 (t)| in realtion to kvk?. By the Cauchy-Schwarz inequality,
|g 000 (t)|
1/2
n
X
vi2 vj2 vk2
i,j,k=1

2 1/2
n
X

f (x0 + tv)
xi xj xk
i,j,k=1

2 1/2

n
X
= kvk3
f (x0 + tv)
xi xj xk
i,j,k=1
Now pick any r > 0 such that Br (x0 ), the closed ball of radius r centered on x0 , is contained in
U , and define Cf,r,x0 by
Cf,r,x0

2 1/2
n
X
= max
f (x)
: x Br (x0 )
i,j,k=1 xi xj xk
Thus, we finally have that for all kvk < r, and all |t| 1,
|g 000 (t)| Cf,r,x0 kvk3 .
(6.8)
Then Cf,r,x0 < by the fundamental theorem on the existence of maximizers for continuous
functions on closed bounded sets. In the next subsection, we apply (6.8) to control the remainder
term in a Taylor exapnsion.
6.1.2
A multivariable Taylor expansion
Recall the single variable Taylors Theorem With Remainder, at second order:
1
1
g(t) = g(0) + g 0 (0)t + g 00 (0)t2 + g 000 (c)t3
2
6
(6.9)
239
where c is some number between 0 and t. Provided g 000 is bounded on some neighborhood of t = 0,
the approximation
1
g(t) g(0) + g 0 (0)t + g 00 (0)t2
(6.10)
2
will be a good approximation for small values of t, since then t3 is small compared to t2 and t.
Our goal in this subsection is to promote (6.9) and (6.10) to multivariable formulas. To do this,
pick any x0 Rn and any x that is close to x0 , so that kx x0 k is small. Consider a function
f : Rn R such that all of the third derivatives of f exist and are continuous.
1
(x x0 ) which is a unit vector. Then define
Define v =
kx x0 k
g(t) := f (x0 + tv)
(6.11)
as usual. Note that by the definition of v,

x0 + kx x0 kv = x0 + (x x0 ) = x ,
(6.12)
and hence g(kx x0 k) = f (x). Therefore, if we apply (6.11) with

t = kx x0 k
and
v=
1
(x x0 ) ,
kx x0 k
(6.9) becomes
1
1
f (x) = f (x0 ) + g 0 (0)t + g 00 (0)t2 + g 000 (c)t3 .
2
6
We now evaluate the last three terms on the right.
First, g 0 (0) = v f (x0 ), and then since kx x0 kv = x x0 ,
g 0 (0)t = (x x0 ) f (x0 ) .
Likewise by (6.5),
g 00 (0)t2
= kx0 xk2 v [Hessf (x0 )]v

=
(x x0 ) [Hessf (x0 )](x x0 ) ,
where in the last line we have used kx x0 kv = x x0 again. Finally, by (6.8),

|g 000 (c)|t3 Cf,r,x0 kvk3 kx x0 k3 = Cf,r,x0 kx x0 k3 .
Therefore, from (6.9) with t = kx x0 k, we have

f (x) f (x0 ) + (x x0 ) f (x0 ) + 1 (x x0 ) [Hessf (x0 )](x x0 )

2
1
Cf,r,x0 kx x0 k3 . (6.13)
6
This brings us to:
Definition 64 (Best quadratic approximation). Let f be a function defined on an open set U
Rn with values in R such that every second order partial derivatives of f exists and is continuous
everywhere in U . For any x0 U , define the function h(x) by
1
h(x) = f (x0 ) + (x x0 ) f (x0 ) + (x x0 ) [Hessf (x0 )](x x0 ) .
2
Then h(x) is the best quadratic approximation to f at x0 .
(6.14)
240

The use of best will be justifeid on Theorem 62 below. The best quadratic approximation to
f at x0 belongs to the class of class of quadratic functions:

Definition 65 (Quadratic function). A quadratic function f on Rn is a polynomial in (x1 , . . . , xn )
each term of which is of degree at most two. A purely quadratic function f on Rn is a polynomial in
(x1 , . . . , xn ) each term of which is of degree exactly two.
According to the definition, the general quadratic function has the form
f (x) = c +
n
X
n
X
bj xj +
j=1
Ai,j xi xj
i,j=1
for some coefficients c, bj , Ai,j , 1 i, j n. We can express this in vector notation as

f (x) = c + b x + x Ax
for some c R, b Rn and A an n n matrix. Since
x Ax =
n
X
xi xj Ai,j
j=1
the value of x Ax is not changed if we replace A by the symmetric matrix with entries
1
(Ai,j + Aj,i ) .
2
This brings us to the following definition:
Definition 66 (Transposes and symmetric matrices). Let B be any m n matrix. The transpose
of B is the n m matrix B T with (B T )i,j = Bj,i . A matrix B , necessarily square, is symmetric in
case B = B T .
The following simple theorem turns out to be very useful:
Theorem 61 (The dot product and the transpose). Let B be any m n matrix. Then for all vectors
x Rn and y Rm ,
y Bx = (B T y) x .
(6.15)
Proof: We compute
m X
n
n X
m
m
n
X
X
X
X
y Bx =
yi
Bi,j xj =
yi Bi,j xj =
yi Bi,j xj =
i=1
j=1
i=1 j=1
j=1 i=1
n
X
j=1
xj
m
X
!
T
Bj,i
yi
= (B T y) x .
i=1
The following lemma tells us, in particular, that if A is a symmetric matrix u Au = 0 for all
unit vectors u, then A = 0; i.e., A is the zero matrix. This would be false without the assumption
that A is symmetric. Consider the 2 2 matrix
"
A=
#
.
(6.16)
241
The matrix A represents counterclockwise rotation through the angle /2 in the plane R2 . Hence
Ax is orthogonal to x for all x in R2 ; yet A is not the zero matrix.
Lemma 14. Let A be a symmetric matrix, and define the function (u) := u Au on the unit sphere;
i.e., the set of all unit vectors. Suppose that is constant on the unit sphere; i.e., (u) = (v) for
all unit vecotrs u and v. Then A is a multiple of the identity matrix, and the multiple is the constant
value of .
Proof: For any i 6= j, consider the unit vector u = 21/2 (ei + ej ), and the unit vectors ei and ej .
Then a simple computation shows
0 = 2u Au ei Aei ej Aej = 2(ei Aej + ej Aei ) = 2(Ai,j + Aj,i ) = 2Ai,j ,
since A is symmetric. Thus, every off-diagonal entry of A is zero.
But since Ai,i = e1 Aei = ej Aej = Aj,j , each of the diagonal entries is the same, and so A is
a (e1 ) times the identity matrix.
The class of quadratic functions has some useful invariance properties.
Lemma 15. Let f (x) be a quadratic function on Rn . Then for any x0 Rn , the function g(e
x)
defiend by
g(e
x) = f (e
x + x0 )
e.
is a quadratic function of x
Proof: Let f = c + b x + x Ax, with A symmetric. Then
g(e
x)
= c + b (e
x + x0 ) + (e
x + x0 ) A(e
x + x0 )
=
e+x
eA x
e
[c + b x0 x0 Ax0 ] + [b + 2Ax0 ] x
ex
e=x
e Ae
= e
c+b
x,
e = [b + 2Ax0 ].
where e
c = [c + b x0 x0 Ax0 ] and b
Theorem 62 (Best fit property of the quadratic approximation). Let x0 Rn . Let f be any function
from Rn to R such that for some r > 0, all of the third order partial derivatives of f exists and are
continuous at each z with kz x0 k r. Let h(x) be the quadratic approximation to f (x) at x0 . Then
lim
xx0
|f (x) h(x)|
=0.
kx x0 k2
(6.17)
Moreover, h(x) is the only quadratic function with this property: If q(x) is any quadratic function
such that
lim
xx0
|f (x) q(x)|
=0,
kx x0 k2
then q(x) = h(x) for all x.

Proof: By (6.13), for kx x0 k r,
|f (x) h(x)|
1
kx x0 kCf,r,x0
2
kx x0 k
6
(6.18)
242
As x approaches x0 , the right side approaches zero, and so by the squeeze principle, (6.17) is proved.
To see that h is the unique quadratic function with this property, suppose that q is another one,
and note that since |q(x) h(x)| |q(x) f (x)| + |f (x) h(x)|, whenever (6.18) is true, it follows
that
lim
xx0
|q(x) h(x)|
=0.
kx x0 k2
(6.19)
Now, the difference of two quadratic functions is quadratic, so h q is a quadratic function of

e = x x0 ,
x, But then by Lemma 15, if we define x
g(e
x) = (h q)(e
x + x0 ) = (h q)(x) ,
(6.20)
e, and (6.19) becomes

g is a quadratic function of x
|g(e
x)|
=0.
e0 ke
x
xk2
lim
(6.21)
ex
e+x
e Ae
Since g is quadratic, g(e
x) = e
c+b
x. Evidently, limxe0 g(e
x) = e
c, and so (6.21) is
e
e
e which
e+x
e Ae
e = tb,
impossible unless e
c = 0. Hence g(e
x) = b x
x. Suppose that b 6= 0. Then take x
goes to 0 at t goes to 0. With this choice,
e 2 + t2 B
e
e 2 + tB
e
e Ab
e Ab
g(e
x)
tkbk
kbk
=
=
.
2
2
2
2
e
e
ke
xk
t kbk
tkbk
e = 0, and so g(e
e Ae
This does not go to 0 as t goes to 0. Hence it must be the case that b
x) = x
x.
e = tv, which goes to 0 at t goes to 0, we have
But then for any unit vector v Rn , taking x
g(e
x)
t2 v Av
v Av
=
=
2
2
2
ke
xk
t kvk
kvk2
which is independent of t. Hence (6.21) is impossible unless v Av = 0 for all unit vectors v. By
Lemma 14, this means A = 0, and hence g(e
x) = 0 for all x. Then by (6.20), (h q)(x) = 0 for all x.
This proves that q(x) = h(x), and thus that the best quadratic approximation is unique.
6.1.3
Clairaults Theorem
The following theorem may at first appear to be a mere curiosity, but in fact, we shall make extensive
use of it in studying the geometry of curvature that is encoded into the various second order directional
derivatives.
Theorem 63 (Clairaults Theorem). f (x, y) be a function of the two variables x and y such that all
partial derivatives of f order 2 are continuous in a neighborhood of (x0 , y0 ). Then

f (x0 , y0 ) =
f (x0 , y0 ) .
x y
y x
Proof: The basic idea of the proof is to take some small h > 0, and then compute a formula
f (x0 + h, y0 + h) f (x0 , y0 )
(6.22)
243
in terms of the mixed second order partial derivatives of f two different ways: First, we keep track
of how f (x, y) changes along the path going straight from (x0 , y0 ) to (x0 + h, y0 ), and then straight
from (x0 + h, y0 ) to (x0 + h, y0 + h). Second, we keep track of how f (x, y) changes along the path
going straight from (x0 , y0 ) to (x0 , y0 + h), and then straight from (x0 , y0 + y) to (x0 + h, y0 + h).
Lets begin: By the Fundamental Theorem of Calculus,
Z h
f (x0 + t, y0 )dt
f (x0 + h, y0 ) f (x0 , y0 ) =
x
0
and
Z
f (x0 + h, y0 + h) f (x0 + h, y0 ) =
0
f (x0 + h, y0 + t)dt .
y
Together we have
f (x0 + h, y0 + h) f (x0 , y0 ) =
Z
0
f (x0 + t, y0 )dt +
x
f (x0 + h, y0 + t)dt .
y
(6.23)
Going along the other path we have

h
Z
f (x0 , y0 + h) f (x0 , y0 ) =
0
and
Z
f (x0 + h, y0 + h) f (x0 , y0 + h) =
0
f (x0 , y0 + t)dt
y
h
f (x0 + t, y0 + h)dt .
x
Together we have
f (x0 + h, y0 + h) f (x0 , y0 ) =
Z
0
f (x0 + t, y0 + h)dt +
x
Z
0
f (x0 + h, y0 + t)dt .
y
(6.24)
The since (6.23) and (6.24) give two different formulas for the same thing, the differences between
the right hand sides must be zero. After a bit of algebra, we conclude that

Z h
f (x0 + h, y0 + t)
f (x0 , y0 + t) dt =
y
y
0

Z h
f x0 + t, y0 + h)
f (x0 + t, y0 ) dt .
x
x
0
By the Mean Value Theorem, for some 0 t1 h
Z
0
f (x0 + h, y0 + t)
f (x0 , y0 + t) dt =
y
y

h
f (x0 + h, y0 + t1 )
f (x0 , y0 + t1 ) .
y
y
Now on the right hand side only x changes, and by the Mean Value Theorem once more, this time
applied to the variation in x, for some 0 s1 h,

h
f (x0 + h, y0 + t1 )
f (x0 , y0 + t1 ) = h2
f (x0 + s1 , y0 + t1 ) .
y
y
x y
244
In summary, for some 0 s1 , t1 h,

1
f (x0 + s1 , y0 + t1 ) = 2
x y
h
Z
0
f (x0 + h, y0 + t)
f (x0 , y0 + t) dt .
y
y
(6.25)
In exactly the same way we deduce that for some 0 s2 , t2 h

Z h

f (x0 + s2 , y0 + t2 ) = 2
f (x0 + t, y0 + h)
f (x0 + t, y0 ) dt .
y x
h 0
x
x
(6.26)
Combining (6.25), (6.25) and (6.26), we see that for some 0 s1 , t1 h and some 0 s2 , t2 h,

f (x0 + s1 , y0 + t1 ) =
f (x0 + s2 , y0 + t2 ) .
x y
y x
Now take the limit h 0, along which s1 , s2 t1 , t2 all tend to zero. By the continuity of the second
order partial derivatives, we conclude that (6.22) is true.
Theorem 64 (Symmetry of the Hessian). f be a function defined on an open set U Rn with values
in R. Suppose that all partial derivatives of f order 2 are defined and continuous in U . Then for all
x U , [Hessf (x)] is a symmetric n n matrix.
2
2
f (x) and
f (x) we are only concerned with the two variables
xi xj
xj xi
xi and xj , so this is an immediate consequence of Clairaults Theorem.
Proof: When we compute
We next present one of the most important theorems in linear algebra. This theorem will, on
account of the symmetry of the Hessian, always give us a preferred coordinate system in which the
best quadratic approximation takes on a very simple form.
6.1.4
Eigenvalues and eigenvectors of a symmetric matrix
Definition 67 (Eigenvectors and eigenvalues). Let A be any n n matrix. A number is an

eigenvalue of A in case there is some non-zero vector v such that
Av = v .
In this case, the vector v is an eigenvector of A.
There are many kinds of problems involving an n n matrix in which it is very helpful to find
all of the eigenvectors of A. This is because A has a very simple effect on eigenvectors: If v is an
eigenvector of A, Av is simply a scalar multiple of v.
"
Some nn matrices have no eigenvectors at all: Consider the 22 rotation matrix A =
Then for any non-zero v, Av is orthogonal to v, and hence Av cannot be a multiple of v.

However, as the following theorem says, if A is symmetric, there will always be eigenvectors,
and plenty of them: If A is an n n matrix, there exists an orthonormal basis {u1 , . . . , un } of Rn
consisting of eigenvectors of A. Using coordinates based on this orthonormal basis, one can easily
answer questions about how the graph of z = f (x) curves away from its tangent plane, as we shall
see. It is for this purpose that we present this theorem now, but as we shall see later on, it has many
other uses. It is one of the most important theorems in linear algebra.
245
Theorem 65 (The Spectral Theorem). Let A be any n n symmetric matrix. Then there is an
orthonormal basis {u1 , . . . , un } of Rn and a set of n real numbers {1 , . . . , n } such that
Auj = j uj
for each j = 1, . . . , n. That is, the orthonormal basis {1 , . . . , n } consists of eigenvectors of A.
The set of numbers {1 , . . . , n } is called the Spectrum of A, hence the name of the theorem.
Rn , define
One way we shall use it is as follows: Given x
un )
y = (y1 , . . . , yn ) = (
x u1 , . . . , x
with respect to the orthonormal basis {u1 , . . . , un }. That is,
so that y is the coordinate vector of x
=
x
n
X
yj uj .
j=1
Therefore,
A
x=
n
X
yj Auj =
n
X
j=1
and hence
A
x
x=
n
X
yj j uj ,
j=1
n
n
X
X
yj uj
yj j uj =
j yj2
j=1
j=1
j=1
A
which displays x
x as a sum of squares.
This can be applied to see how the graph of z = f (x) curves away from its tangent plane:
Suppose that x0 is a critical point of a continuously twice differentiable function f on Rn . Let
= x x0 , the best quadratic approximation of f at x0 is
A := [Hessf (x0 )]. Then for x
1
) = f (x0 ) + x
A
f (x0 + x
x.
2
In the coordinate system based on the orthonormal basis {u1 , . . . , un }, consisting of eigenvectors of
A with eigenvalues {1 , . . . , n }, we can write the right hand side as
n
1X
f (x0 ) +
j yj2 .
2 j=1
Written in this form, it is easy to understand the curvature of the graph of xn+1 = f (x) near
x0 (and in particular, the graph of z = f (x, y) for n = 2). If all of the eigenvalues j are strictly
positive, then the graph is curving upwards from its height at x0 . If all of the eigenvalues j are
strictly negative, then the graph is curving downwards from its height at x0 . And if the signs are
mixed, it is curving upwards in some directions, and downwards in others. Moreover, as we shall see,
the eigenvectors tell us in which directions the graph curves up and in wich directions the graph
curves down.
To understand how the the graph of z = f (x) curves away from its tangent plane at x0 , it suffices
to find the eigenvalues and eigenvectors of [Hessf (x0 )].
246

We shall return to this application of the Spectral Theorem in the next section. In the rest of
this one, we prove the theorem, and explain how to compute the eigenvectors and eigenvalues of a
symmetric matrix. We begin with some lemmas.
Lemma 16 (Orthogonality lemma). Let A be a symmetric matrix, and suppose that and are
two distinct eigenvalues of A, and that Av = v and Aw = w. Then v and w are orthogonal.
Proof: By Theorem 6.15 and the definitions
w v = w (v) = w Av = Aw v = w v .
Therefore, 0 = ( )w v, and since 6= , w v = 0
The next lemma is the heart of the matter:
Lemma 17 (Maximum values and eigenvalues). The function (x) defined by
(x) = x Ax
has a maximizer u1 on the unit sphere S n1 := {x : kxk = 1 }. The maximum value 1 := (u) is
an eigenvalue of A, and the maximizer u1 is an eigenvector with this eigenvalue:
Au1 = 1 u1 .
Proof: The function is a continuous function on the unit sphere S n1 , and the unit sphere is
closed and bounded. Thus, has a maximizer on S n1 , meaning that there is a number 1 and a
unit vector u1 such that
1 = (u1 ) (x)
for all
u S n1 .
(6.27)
To see that u1 is an eigenvector of A, l with eigenvalue 1 let v be any unit vector orthogonal
to u1 , and t R. Then ku1 + tvk2 = 1 + t2 , and so
u1 + tv
u(t) :=
1 + t2
is a unit vector for each t Note that u(0) = u1 . It follows from (6.27) that
1 = (u(0)) (u(t))
for all t R ,
and hence the derivative of (u(t)) must be zero at t = 0. We compute, using Theorem 6.15
d
(u(t)) = u0 (t) Au(t) + u(t) Au0 (t) = 2u0 (t) Au(t) ,
dt
and that
u0 (t) =
1
t
v
(u1 + tv) .
(1 + t2 )1/2
(1 + t2 )3/2
Thus,

d
0 = (u(t))
= 2v Au1 ,
dt
t=0
(6.28)
247
and this is true for each v that is orthogonal to u1 . But if Au1 is orthogonal to every vector orthogonal
to u1 , it must be a multiple of u1 . (This follows by expanding Au1 in any orthonormal basis for Rn
whose first vector is u1 .) Thus Au1 = u1 for some number . To determine , that the dot product
of both sides with u1 to see that u1 Au = . The left hand side is (u1 ) = 1 , and so = 1 . Thus:
Au1 = (u1 Au1 )u1 = 1 u1 ,
and u1 is indeed an eigenvector with eigenvalue 1 .
Proof of the Spectral Theorem: Suppose A is an n n symmetric matrix, and we have found
an orthonormal set {u1 , . . . , um } of m < n eigenvectors of A. Let {1 , . . . , k } be the corresponding
sequence of eigenvalues. By Lemma 17, we can always find such a set with m = 1 at least. We
shall now show that we can always enlarge any such set into an orthonormal basis of Rn consisting
of eigenvectors of A. The first step is to modify A so that we can apply Lemma 17 to get a new
eigenvector that is not already in our given set {u1 , . . . , um }.
Consider the linear transformation g from Rn to Rn given by
g(x) =
m
X
k (uk x)uk .
k=1
Let B denote the corresponding matrix:

Bi,j = ei Bej =
m
X
k (uk ei )(uk ej ) .
k=1
Note that this is a symmetric matrix. Note also that

Buk = k uk
for each j = 1, . . . , m ,
(6.29)
and
Bu = 0
whenever u is orthogonal to uk for each k = 1, . . . , m .
(6.30)
Now consider the symmetric matrix C := A B, and consider the function (u) := u Cu on
the unit sphere. If is identically zero on the unit sphere, then Lemma 14 says that C is the zero
matrix, and so A = B. In this case we are finished, since if {u1 , . . . , un } is any orthonormal basis
of Rn extending {u1 , . . . , un }, (6.29) says that Buk = k uk for 1 k m, and (6.30) says that
Buk = 0 = 0uk for m + 1 k n. Hence {u1 , . . . , un } is an orthonormal basis of Rn consisting of
eigenvectors of B = A.
It remains to consider the case in which is not identically zero. Then it has a strictly positive
maximum, or a strictly negative minimum. Let us suppose it has a strictly positive maximum which
we denote by m+1 . Then by Lemma 17, there is a unit vector um+1 of C such that
Cum+1 = m+1 um+1 .
By (6.29), for 1 k m,
Cuk = Auk Buk = k uk k uk = 0 ,
(6.31)
248
and hence each vector in {u1 , . . . , um } is an eigenvector of C with eigenvalue 0. Then by Lemma 16,
um+1 is orthogonal to each vector in {u1 , . . . , um }. From this and (6.30), it follows that Bum+1 = 0.
Hence (6.31) becomes
Aum+1 = m+1 um+1 .
Thus, we have extended {u1 , . . . , um } to an orthonormal set of m + 1 eigenvectors of A. If n = m + 1,
this is a basis of Rn . If not, we can repeat the procedure and enlarge the orthonormal set until it
contains n vectors and is therefore an orthonormal basis.
Now that we know bases of eigenvectors of symmetric matrices exist, how do we find them? This
is not hard for n = 2 or n = 3, and we shall now explain a general method that is applicable for all
n, but is usually only feasible for small values of n.
If v 6= 0, and Av = v, then (AI)v = 0. Of course (AI)0 = 0, so the linear transformation
corresponding to A I is not invertible. Conversely, we have seen that whenever A I is not
invertible, the corresponding linear transformation is not one-to-one, and so there is a non-zero vector
v such that (A I)v = 0. Thus, is an eigenvalue of A if and only if A I is not invertible.
We have proved:
Theorem 66 (Eigenvalues and invertibility). Let A be an n n matrix. Then is an eigenvalue of
A if and only if A I is not invertible.
Notice that this theorem is valid whether of not A is symmetric. As we have seen for n = 2 and
n = 3, A I is not invertible if and only if det(A I) = 0. Hence, to find all of the eigenvalues of
a 2 2 or 3 3 matrix A, compute
p(t) := det(A tI) ,
which will be a polynomial of degree n = 2 or n = 3 in t, and then solve the equation
p(t) = 0 .
The solutions are the eigenvalues of A.
"
Example 94 (Eigenvalues of a 2 2 matrix). Let A =
"
det (A tI) = det
2t
2t
#
. Then
#!
= t2 4t + 3 .
The two roots of the quadratic polynomial are t = 3 and t = 1. Thus the eigenvalues are
1 = 3
and
2 = 1 .
Now that you have the eigenvalues, finding the eigenvectors is really easy, at least for n = 2
or n = 3. Indeed, suppose is an eigenvalue of A. Form the matrix A I. We must then find
a non-zero vector v such that (A I)v = 0. By the dot product formulation of matrix-vector
multiplication, this is equivalent to v being orthogonal to each row of A I.
249
In particular, if n = 2, and the first row of A I is (a, b) 6= (0, 0), then v must be a multiple
of (a, b) = ( b, a). The same reasoning applies to the second row. If both rows are zero, it means
A = I, so every non-zero vector is an eigenvector of A.
Moreover, when the two eigenvalues are distinct, Lemma 16 say the second eigenvalue must be
orthogonal to the first, so you do not need to solve for the second eigenvector once you have the first
any non-zero orthogonal vector will do.
"
Example 95 (Eigenvectors of a 2 2 matrix). Let A =
#
. We have seen above that the two
eigenvalues are 1 = 3 and 2 = 1. Form

"
A 3I =
The first row is ( 1, 1) so ( 1, 1)
#
.
= ( 1, 1) is an eigenvector with eigenvalue 3, as you can
check. Of course, so is any non-zero multiple of this, so (1, 1) is also an eigenvector with eigenvalue
3.
To get the second eigenvector, we simply use Lemma 16 which says that this must be orthogonal
to the one we have already found. You can even use the first row of A 3I, so we do not even have to
write down A I. Thus, as you can check, ( 1, 1) is an eigenvector with eigenvalue 1. Normalizing
these, we get our orthonormal basis:

{u1 , u2 } =
1
1
(1, 1) , ( 1, 1)
2
2

.
The situation for n = 3 is is not much more complicated. If v is orthogonal to each row of
A I, and any two of these rows are not multiples of one another, v must be a multiple of the cross
product of these two rows. On the other hand, if all of the rows are mutiples of the same vector r,
we know how to get an orthonormal basis {u1 , u2 , u3 } such that u1 is a multiple of r. Then u2 and
u3 are eigenvectors with eigenvalue .
The challenge in n = 3 is in finding the eigenvalues, since this involves solving for the roots of a
cubic polynomial.
6.1.5
Principle curvatures at a critical point for functions of 2 variables
Let f : R2 R be such that all of the second order partial derivatives of f are continuous, and
suppose that x0 is a critical point of f . Thus, f (x0 ) = 0, and so the quadratic approximation of f
at x0 is
1
[Hessf (x0 )]
f (x0 ) + x
x,
2
:= x x0 .
where as before, we define x
(6.32)
Let {u1 , u2 } be an orthonormal basis of R2 consisting of eigenvectors of [Hessf (x0 )], with 1 and
2 being the corresponding eigenvalues. Define
u = u1 x
and
v = u2 x
with respect to the {u1 , u2 } basis.

so that (u, v) are the coordinates of x
250

= uu1 + vu2 ,
Then, since x
[Hessf (x0 )]
x = [Hessf (x0 )](uu1 + vu2 ) = u1 u1 + v2 u2 .
Therefore,
[Hessf (x0 )]
x
x = 1 u2 + 2 v 2 ,
and so in the new coordinate system, the quadratic approximation of f at x0 is simply
1
1
f (x0 + uu1 + vu2 ) f (x0 ) + 1 u2 + 2 v 2 ,
2
2
(6.33)
Example 96 (Quadratic approximation at a critical point). Let

f (x, y) = x3 + y 3 3xy .
We have seen in Example 92 that ( 1, 1) is a critical point, and that
#
"
2 1
.
[Hessf (x0 )] = 3
1 2
"
are 3 and 1, so the two

1 2
eigenvalues of [Hessf (x0 )] are 9 and 3. One easily computes f (1, 1) = 1.
We have seen in Example 94 that the two eigenvalues of
Therefore, the quadratic approximation to f (x, y) near ( 1, 1) is

9
3
f (( 1, 1) + uu1 + vu2 ) 1 u2 v 2 .
2
2
The level curves of the quadratic function on the right are ellipses in the u, v plane. We will
soon see how to use this infrmration together with the specification of the eigenvectors of [Hessf (x0 )]
to sketch the level curves of f itself near x0 .
First though, we learn very useful information from the eigenvalues alone: Since in this example
they are both negative, f (x0 ) is strictly higher than f (x) for all x in a neighborhood of x0 : The surface
z = f (x, y) curves strictly downward in every direction at x0 . That is, x0 is a local maximizer of
f.
Moreover, the direction in which the curvature is most negative is the direction along the u-axis,
while the direction in which the curvature is least negative is the direction along the v-axis. These
are the directions of principle curvature of f at the critical point ( 1, 1).
What we saw in Example 96 turns out to be generic. First, let us make a general definition.
Definition 68. Let f : R2 R be such that all partial derivatives of f of order 2 exists and are
continuous in a neighborhood of x0 = (x0 , y0 ). Then the principle curvatures of f at are the numbers

2

d
2
2

+ = max
f
(x
+
tu,
y
+
tv)
:
u
+
v
=
1
0
0

dt2
t=0
and

= max

d2
2
2

f (x0 + tu, y0 + tv)
: u +v =1
.
dt2
t=0
251
Let f : R2 R be such that all partial derivatives of f of order 2 exists and are continuous at
x0 . The principle curvatures + and , provided they are not zero, determine the way in which the
curved surface given by z = f (x, y) curves away from the tangent plane to the graph of f at x0 .
Here is an illustrative graph:
15
10
5
0
5
10
3
2 1
15
10
5
0
5
10
3 2 1
As you see, the graph of z = f (x, y) curves up away from the tangent plane at x0 . This means
that for all unit vectors v = (u, v),

d2
f (x0 + tv)
>0.
2
dt
t=0
In particular, both + and are strictly positive. Conversely, if + > 0 and < 0, the surface
z = f (x, y) curves upward in some directions, and downward in others. Here is a graph for another
function that illustrates this:
252
This surface curves upward from the tangent plane at x0 in one direction, and downward in
another.
The same sort of reasoning shows that the curved surface given by z = f (x, y) curves downward
from the tangent plane at x0 in case both + and are strictly negative.
Example 97 (Computing principle curvatures). Let
f (x, y) = 2yx2 + 4x2 + xy + 8y 2 + 8x y .
Then
f (x, y) = (4xy + 8x + y + 8, 2x2 + x 1 + 16y) .
At a critical point (x, y), we must have
4xy + 8x + y + 8
2x2 + x 1 + 16y
From the first equation we see that x =
y+8
. Substituting this into the second, we see, after
4y + 8
some algebra, that

y(476 + 503y + 128y 2 ) = 0 .
This has three roots, one of course being y = 0. The corresponding x value is 1. Hence one of the
three critical points is x0 := ( 1, 0). Let us compute the principle curvatures at x0 , and sketch a
contour plot for f near x0 .
We compute:
2
f (x, y) = 8 + 4y
x2
2
f (x, y) = 1 + 4x
xy
and
Evaluating at x0 = ( 1, 0), we find

"
[Hessf (x0 )] =
16
#
.
The quadratic equation we must solve to find the eigenvalues is

(8 t)(16 t) = 9 ,
2
f (x, y) = 16 .
y 2
253
which reduces to t2 24t = 119. Completing the square, (t 12)2 = 25, and so the two eigenvalues
are
1 = 17
and
2 = 7 .
1
We then get u1 by normalizing (3, 8 17) = 3(1, 3) which yields u1 = (1, 3). Once we
10
1
have u1 , we obtain u2 from u2 = (u1 ) : u2 = (3, 1).
10
In the corresponding coordinate system, the best quadratic approximation to f at x0 is
1
f (( 1, 0) + uu1 + vu2 ) 4 + (17u2 + 7v 2 ) .
2
Let us make a contour plot of the right hand side in the u, v plane. Setting
17u2 + 7v 2 = c ,
we get the equation of an ellipse whose major axis is the v-axis, and whose minor axis is the u-axis:
Here is a plot in the u, v plane for 5 values of c:
Now let us shift and rotate this plot so it fits into the original x, y plane. The point x0 is the
center of the u, v coordinate system, and the positive u axis runs along the u1 direction and the
positive v-axis runs along the u2 direction.
So shifting and rotating the ellipses accordingly, we get our plot:
Here is a contour plot of the actual function f (x, y) over the range 2 x 0, 1 y 1.
254
As you can see, the contour plot corresponds quite closely to the contour plot for the quadratic
approximation over this whole region, which is not even so small. If we look at a larger region, we
see that the approximation breaks down farther away from ( 1, 0):
Indeed, you see there is another critical point near (4, 3/2), and that the contour lines near this
critical point are hyperbolas. What is impressive however, is the size of the region around the critical
point where there contour plots of f and its quadratic approximation are virtually indistinguishable.
Example 98 (Computing principle curvatures). Let f (x, y) = 3x2 y 6x y 3 . Then
f (x, y) = (6xy 6, 3x2 3y 2 ) .
We have a critical point if and only if xy = 1 and x2 = y 2 . The latter equation says x = y. But if
x = y, the xy = 1 is impossible,so x = y, and then we must have x = 1 or x = 1. Hence the two
critical points are
( 1, 1)
and
(1, 1) .
Let us first take x0 = (1, 1). We compute:

2
f (x, y) = 6y
x2
2
f (x, y) = 6x
xy
and
2
f (x, y) = 6y .
y 2

"
Evaluating at x0 := (1, 1), we find [Hessf (x0 )] =
255
.
6 6
The quadratic equation we must solve to find the eigenvalues is
(6 t)(6 t) = 36 ,
which reduces to t2 = 72. The two roots are
1 = 6 2
and
1 = 6 2 .
We then get u1 by normalizing (6, 6 6 2) = 6(1, 1 2) which yields

u1 =
1
(1, 1 2) .
42 2
Once we have u1 , we obtain u2 from u2 = (u1 ) :

u2 =
1
( 2 1, 1) .
42 2
Since f (x0 ) = 2, the best quadratic approximation near x0 in the u, v coordinates is
f ((1, 1) + uu1 + vu2 ) = 2 + 3 2u2 3 2v 2 .

The equation obtained by setting the right hand side is equal to a constant is equivalent to the
equation
u
2 v2 = c ,
for some other constant c, which is the equation of an hyperbola. Here is a plot in the u, v plane for
6 values of c:
Now let us shift and rotate this plot so it fits into the original x, y plane. The center of the u, v
coordinate system is at x0 , the positive u-axis runs along the u1 direction, and the positive v-axis
runs along the u2 direction. Shifting and rotating the hyperbolas accordingly, we obtain:
256
To the extent that the quadratic approximation is valid, this should be a good match to the contour
plot of the original function f (x, y) near to the critical point ( 1, 1). For comparison, here is a
computer generated contour plot of the actual function f (x, y) over the range 2 x, y 0.
As you can see, the actual contour plot corresponds quite closely to the contour plot for the
quadratic approximation over this whole region, which is not even so small. If we look at a larger
region, we see that the approximation breaks down farther away from ( 1, 1):
257
Indeed, you see there is another critical point at ( 1, 1), and that the contour lines near this
critical point are also hyperbolas. What is impressive however, is the size of the region around the
critical point where there contour plots of f and its quadratic approximation are virtually indistinguishable.
6.1.6
Types of critical points for real valued functions on Rn
The best quadratic approximation is particularly significant at a critical point. Suppose that f is
function from Rn to R such that all of the third order partial derivatives exist and are continuous.
Then when x0 is a critical point of f , the best affine approximation of f near x0 , which is f (x)
f (x0 ) + f (x0 ) (x x0 ), reduces to
f (x) f (x0 ) .
There is no x dependence on the right hand side. As we have seen, critical points might be minimizers,
they might be maximizers, and they might be something else altogether. To decide what sort of
critical point x0 might be, we need to look beyond the best affine approximation, and look at the
best quadratic approximation. The following definition singles out two types of critical points that
are of particular interest in applications.
Definition 69 (Local maximizers, minimizers, and saddle points). Let f : Rn R be continuously
differentiable. Then x0 Rn is a strict local maximizer of f in case there is some r > 0 so that
0 < kx x0 k < r
f (x0 ) > f (x) .
We say that x0 is a strict local minimizer of in case there is some r > 0 so that
0 < kx x0 k < r
f (x0 ) < f (x) .
Finally, we say that x0 is a saddle point of f in case x0 is a critical point of f , and for every r > 0
there exist points y and z with ky x0 k < r and kz x0 k < r such that
f (y) < f (x0 ) < f (z) .
Notice that while the definition stipulates that saddle points are critical points, it does not
stipulate that strict local maxima and minima are critical points. Nonetheless, strict local maxima
and minima will be critical points. Indeed, of x0 is a strict local maximum, then x0 is a maximizer
of f on the ball of radius r/2 about x0 for some r > 0. Since interior maxima must be critical points,
x0 is a critical point.
Theorem 67 (Criteria for local maxima and minima). Let f : Rn R. Suppose that x0 is a critical
point of f , and that for some R > 0, all of the third order partial derivatives of f exists and are
continuous at each z with kz x0 k R. Then:
(1) If all of the eigenvalues of [Hessf (x0 )] are strictly negative, x0 is a strict local maximizer of f .
(2) If all of the eigenvalues of [Hessf (x0 )] are strictly positive, x0 is a strict local minimizer of f .
(3) If some eigenvalues of [Hessf (x0 )] are strictly positive, and some are strictly negative, x0 is a
saddle point.
258
Proof: Suppose that all of the eigenvalues j are strictly negative. then for some c > 0,
j c < 0 .
Because of our assumption on third derivatives, we may apply Theorem 62. Let h denote the quadratic
approximation to f at x0 , Since by Theorem 62,
lim
xx0
|f (x) h(x)|
=0,
kx x0 k2
there exists an r > 0 so that whenever 0 < kx x0 k < r, then

c
|f (x) h(x)|
kx x0 k2
4
or, euivalently
|f (x) h(x)|
c
kx x0 k2 .
4
(6.34)
Therefore, whenever 0 < kx x0 k < r,

c
f (x) h(x) + kx x0 k2 .
4
(6.35)
Now let us write out h(x) in convenient notation:

h(x)
1
= f (x0 ) + (x x0 ) [Hessf (x0 )](x x0 )
2
n
1X
= f (x0 ) +
j ((x x0 ) uj )2
2 j=1
n
f (x0 ) c
1X
((x x0 ) uj )2
2 j=1
1
f (x0 ) c kx x0 k2
2
Combining this with (6.35), we find for 0 < kx x0 k < r,
c
f (x) f (x0 ) kx x0 k2 ,
4
and thus for 0 < kx x0 k < r, f (x) < f (x0 ). Thus, x0 is a strict local maximizer. The proof for
strict local minima is entirely analogous.
Finally suppose that i > 0 and j < 0. Pick some number c > 0 so c < i /2 and c < j /2.
Then, once more, let f be such that whenever kx x0 k < r, then (6.34) is true.
Note that
h(x0 + tui ) = f (x0 ) +
t2
i
2
and
h(x0 + tuj ) = f (x0 ) +
t2
j .
2
Thus, from (6.34), since k(x0 + tu1 ) x0 k = t,

f (x0 + tui ) h(x0 + tui )
c t2
c t2
f (x0 ) +
.
44
44
Thus, in every neighborhood of x0 , there are points x with f (x) > f (x0 ).
Likewise,
c t2
c t2
f (x0 )
.
44
44
Thus, in every neighborhood of x0 , there are points x with f (x) < f (x0 ). This shows that x0 is a
f (x0 + tui ) h(x0 + tui ) +
saddle point.
6.1.7
259
Sylvesters Criterion
Definition 70 (Positive definite matrices). Let A be an n n symmetric matrix. Then A is positive

definite in case for every non-zero vector x in Rn ,
x Ax > 0 .
We say A is negative definite if A is positive definite.
Theorem 68 (Eigenvalues and matrix positivity). A symmetric n n matrix A is positive definite
if an only if all of its eigenvalues are strictly positive.
Proof: Let {u1 , . . . , un } be an orthonormal basis of Rn consisting of eigenvectors of A.
Let
{1 , . . . , n } be the corresponding eigenvalues. Since

j = uj Auj ,
it follows that when A is positive definite, then each eigenvalue of A is strictly positive.
On the other hand, suppose that j c > 0 for each j = 1, . . . , n. We can express any x Rn
as
x=
n
X
(x uj )uj ,
j=1
and then, by a familiar computation,

x Ax =
n
X
j (x uj )2 c
j=1
n
X
(x uj ) = ckxk2 .
j=1
Thus, A is positive definite.

We can now restate our conclusions about local minima and maxima as follows: Let x0 be
a critical point of a function f on Rn , all of whose third order partial derivatives are defined and
continuous. Then x0 is a strict local minimum if and only if the Hessian of f at x0 is positive definite,
and x0 is a strict local maximum if and only if the Hessian of f at x0 is negative definite.
The theorem above gives us one way to check whether a Hessian is positive definite or not
compute the eigenvalues. It turns out that there is another way. One can determine whether all of
the eigenvalues are strictly positive simply by by computing certain determinants. We now explain
how this works for n = 2 and n = 3.
"
First, consider the case n = 2. Let A =
and let 1 and 2 be the eigenvalues of A.

c d
Recall that these are the roots of the polynomial
"
#!
at
b
p(t) := det(A tI) = det
= t2 t(a + d) + (ad bc) .
(6.36)
c
dt
On the other and, since 1 and 2 are the roots of p(t), p(t) must be a multiple of (t1 )(t2 ),
and since the coefficient of t2 is one, we have
p(t) = (t 1 )(t 2 ) .
260
Multiplying this out, and comparing with (6.36), we see

t2 (1 + 2 )t + 1 2 = t2 t(a + d) + (ad bc) .
Since this is true for all t, the corresponding coefficients on both sides must be equal, and in particular,
1 2 = ad bc = det(A) .
That is, det(A) > 0 if and only if either both eigenvalues of A are strictly positive, or both both
eigenvalues of A are strictly negative, and det(A) < 0 if and only if one eigenvalue of A is strictly
positive, and the other is strictly negative.
To determine the sign of the eigenvalues when det(A) > 0, remember from the proof of the
Spectral Theorem that the largest and smallest eigenvalues of A are, respectively the maximum and
minimum values of the function (x) := x Ax on the unit sphere. Thus, when both eigenvalues are
strictly positive, x Ax will be strictly positive for every unit vector x, and when both eigenvalues
are strictly negative, x Ax will be strictly negative for every unit vector x. Thus we only need to
look at e1 Ae1 = a to determine the signs. We summarize:
(1) Both eigenvalues are strictly positive if and only if det(A) > 0 and a > 0.
(2) Both eigenvalues are strictly negative if and only if det(A) > 0 and a < 0.
(3) If det(A) < 0 , one eigenvalue is strictly positive, and the other is strictly negative.
This is the 22 version of Sylvesters Criterion. The 33 case is only slightly more complicated.
Let
A=
u
v
"
and
B :=
#
.
Notice that B is the 2 2 block in the upper left corner of A.

The 3 3 version of Sylvesters Criterion states that
(1) All three eigenvalues are strictly positive if and only if det(A) > 0, det(B) > 0 and a > 0.
(2) All three eigenvalues are strictly negative if and only if det(A) < 0, det(B) > 0 and a < 0.
(3) Suppose none of det(A), det(B) and a is zero, but there is any other pattern of signs than specified
in (1) and (2). Then at least one eigenvalue is strictly positive, and at least one eigenvalue is strictly
negative.
To see this, let 1 , 2 and 3 be the eigenvalues of A, and let p(t) = det(A tI). Note that
p(0) = det(A), so the constant term in p(t) is det(A). Arguing as above, the constant term is also
the product of the three eigenvalues. That is,
det(A) = 1 2 3 .
If det(A) > 0, then there are two possibilities: Either each eigenvalue is strictly positive, or else
two are strictly negative, and one is strictly positive. Suppose 1 and 2 are strictly negative and 1
is strictly positive. Let {u1 , u2 , u3 } be a corresponding orthonormal basis of eigenvectors of A.
6.2. CURVATURE OF SURFACES IN R3
261
Consider the plane parameterized by x(s, t) = su1 + tu2 . Since every two planes through the
origin in R3 intersect at least in a line, There is a unit vector in this parameterized plane that also
lies in the x, y plane. That is, for some s and t with s2 + t2 = 1, and some x and y with x2 + y 2 = 1,
su1 + tu2 = (x, y, 0) .
We then compute
(su1 + tu2 ) A(su1 + tu2 ) = 1 s2 + 2 t2 < 0 ,
and
(x, y, 0) A(x, y, 0) = (x, y) B(x, y) .
(6.37)
We conclude that there is a unit vector (x, y) such that (x, y) B(x, y) < 0. This means that
the least eigenvalue of B, which is the minimum value of x Bx as x ranges over the unit circle, is
strictly negative. Hence B has at least one negative eigenvalue.
By the 2 2 version of Sylvesters Criterion, if det(B) > 0 and a > 0, both eigenvalues of B
must be positive. Therefore, when det(A) > 0, det(B) > 0 and a > 0, it is impossible that two of
the eigenvalues of A are negative; they must all be positive.
On the other hand, when all of the eigenvalues of A are strictly positive, so that A is positive
definite, (6.37) implies that B is positive definite also, and hence both of its eigenvalues must be
strictly positive. So in this case det(B) > 0 and a > 0. Finally since det(A) = 1 2 3 , det(A) > 0
when all of the eigenvalues of A are strictly positive.
This proves (1). To prove (2) note that if we replace A by A, det(A) and a change sign, but
det(B) does not. Then (3) follows directly from (1) and (2).
There is an n n version of Sylvesters Criterion, whose form you can probably guess. However,
since have not introduced higher dimensional determinant in this course, we stop here.
6.2
6.2.1
Curvature of surfaces in R3
Parameterized surfaces in R3
Definition 71 (Parameterized surface). A parameterized surface S in R3 is a continuously differentiable function

X(u, v) = (x(u, v), y(u, v), z(u, v))
from an open set U in R2 to R3 such that [DX (u, v)] has rank 2 for each (u, v) U .
Differentiable surfaces in R3 are the two dimensional analogs of differentiable curves x(t) in R3 .
As we shall see below, the meaning of the final condition in the definition is exactly that the surface
S is really two dimensional, and not a one dimensional object, like a curve, in disguise. For example,
let x(t) be any differentiable function from the interval (0, 1) to R3 . Define
X(u, v) := x(uv)
262
on U = (0, 1) (0, 1) R2 . The image of this function is not a surface, but a curve. Moreover, one
computes in this case that
[DX (u, v)] = [vx0 (uv), ux0 (uv)] .
The two columns of [DX (u, v)] are multiples of one another, and so [DX (u, v)] has rank 1.
One example of a parameterized surface is provided by the graph of a continuously differentiable
function f on an open set U R2 . Then we take u = x, v = y and z = f (x, y) = f (u, v), and we
have
X(u, v) = (u, v, f (u, v)) .
Then
[DX (u, v)] =
1
0
f (u, v)
u
.
1
f (u, v)
v
This evidently has rank 2.

Not all parameterized surfaces are graphs: As we have seen early on in Chapter One, (most of)
the unit sphere in R3 can be parameterized by
X(u, v) = ( cos u sin v, sin u sin v, cos v)
for < u < and 0 < v < . Since x(u, v) = x(u, v) and y(u, v) = y(u, v), this is not a graph
the graph of a function of (u, v). The Jacobian matrix is given by
sin u sin v cos u cos v
[DX (u, v)] =

cos u sin v sin u cos v .
0
sin v
Since sin v > 0 for 0 < v < , the two columns of [DX (u, v)] are never multiples of one another,
and so the rank of [DX (u, v)] is 2. Thus the parameterization of the unit sphere that we met in
Chapter One is, as we would expect, a parameterized surface.
Now let X be a continuously differentiable function defined on an open set U R2 with values
in R3 . This is a parameterized surface if and only if [DX (u, v)] has rank 2 for each (u, v) U .
[DX (u, v)] has rank 2 if and only if the columns of [DX (u, v)] are linearly independent, which, since
the columns are vectors in R3 , is the case if and only if the cross product of the columns is not 0.
The two columns of [DX (u, v)] are the vectors
X
(u, v)
u
and
X
(u, v) .
u
X
X
(u, v) and
(u, v) are tangent vectors to curves in the surface S; they are tangent
u
v
to the curves xu (t) := X(t + u, v) and xv (t) = X(u, t + v) respectively, at t = 0. When these two
The vectors
vectors are linearly independent at (u0 , v0 ), the where

n(u, v) :=
X X
(u, v) .
u
v
(6.38)
263
X
is not the zero vector at (u, v) = (u0 , v0 ). Since n(u0 , v0 ) is orthogonal to both
(u0 , v0 ) and
u
X
(u0 , v0 ), the plane given by the equation
v
n(u0 , v0 ) (x X(u0 , v0 )) = 0
(6.39)
is the plane in R3 containing the three points

X(u0 , v0 ) ,
X(u0 , v0 ) +
X
(u0 , v0 )
u
and X(u0 , v0 ) +
X
(u0 , v0 ) .
v
Therefore, it is the plane parameterized by

x(s, t) := X(u0 , v0 ) + s
X
X
(u0 , v0 ) + t
(u0 , v0 ) .
u
v
The right hand side can be witten in matrix form as X(u0 , v0 ) + [DX (u0 , v0 )](s, t). Since X is
differentiable at (u0 , v0 ), for each > 0, there is a > 0 such that
k(u, v) (u0 , v0 )k <
kX(u, v) X(u0 , v0 ) [DX (u0 , v0 )](u u0 , v v0 )k k(u, v) (u0 , v0 )k . (6.40)

Thus for (u, v) in a sufficiently small neighborhood of (u0 , v0 ), X(u, v) is very closely approximated by X(u0 , v0 ) [DX (u0 , v0 )](u u0 , v v0 ). As (u, v) ranges over the small neighborhood of
(u0 , v0 ), this functions traces out a small patch about X(u0 , v0 ) of the plane given by (6.39).
It follows that when the condition that [DX (u, v)] has rank 2 at each (u, v) U is satisfied, S is truly
two dimensional in the sense that S is arbitrarily well approximated by a plane in any sufficiently
small neighborhood of any point in S. These approximating planes are the tangent planes to S.
Definition 72 (Unit normal and tangent plane). Let S be a parameterized surface. The unit normal
b
vector N(u,
v) denotes is given by
b
N(u,
v) =
1
n(u, v) ,
kn(u, v)k
(6.41)
n(u, v) =
X X
(u, v) .
u
v
(6.42)
where
(We put the hat on this unit vector to distinguish it from the unit normal vector to the curve, which
we shall denote by N(t) as usual.) The tangent plane to S at a point X(u0 , v0 ) in S is the plane
given by
b 0 , v0 ) (x X(u0 , v0 )) = 0 .
N(u
Notice that if we changed the order of u and v, the sign of n(u0 , v0 ) would change, thus changing
the orientation of our surface. There is a choice of orientation implicit in (6.42).
b
Here is a picture showing a surface S , and the field of unit normal vectors N(u,
v) for a
number of choices of u and v:
264
Through the base of each arrow, there is the plane orthogonal to the arrow. This is the tangent
plan at that point in the surface. If we graphed the tangent planes themelves, the picture would
be all jumbled up. But by graphing the normal vectors, we can clearly see how that tangent planes
varies over the surface due to the curvature of the surface. In the next subsection we shall make
this notion of curvature of a surface precise. Let us close this subsection by making the connection
between the new notion of a tangent plane to a parameterized surface in R3 and the familiar notion
of the tangent plane to the graph of a function f (x, y) on R2 .
Example 99 (Tangent planes for graphs). Let f (x, y) be a continuously differentiable function on
R2 . Consider the parameterized surface S given by
X(u, v) = (u, v, f (u, v)) .
As we have seen above,
[DX (u, v)] =
1
0
f (u, v)
u
.
1
f (u, v)
v
Therefore,

n(u, v) =
f (u, v)
f (u, v)
,
, 1
u
v

.
Defining X0 = X(u0 , v0 ), we have that n(u0 , v0 ) (x X0 ) = 0 can be written as

f (u, v) f (u, v)
,
, 1 (x X0 ) = 0 ,
u
v
which is the equation of the tangent plane to the graph of z = f (x, y) at (x, y) = (u0 , v0 ). Hence the
new notion of tangent plane agrees with the old one for surfaces that are graphs, as it should.
6.2.2
Principal curvatures and Gaussian curvature
Let S be a surface parameterized by X(u, v) = (x(u, v), y(u, v), z(u, v)) for (u, v) in some open set
U R2 . Suppose that the functions x(u, v), y(u, v), z(u, v) are twice continuously differentiable. Let
(u0 , v0 ) D, and let x0 = X(u0 , v0 ), and note that this is a point on the surface S.
We shall study the curvature of the surface by looking at the curvature of curves in S that pass
through x0 . Let u(t) := (u(t), v(t)) be a differentiable curve in R2 with
u(0) = (u0 , v0 )
and
u0 (0) = (a, b) .
265
Now, using the parameterization, lift this curve in R2 up to a curve

x(t) = X(u(t)) ,
which is a three dimensional curve lying on the surface S.
The speed of this curve, as it passes through x0 is kx0 (0)k. We compute, using the chain rule
and u0 (0) = (a, b),
x0 (0) = [DX (u0 , v0 )]u0 (0) = [DX (u0 , v0 )](a, b) = a
Notice that the vectors
Define
X
X
(u0 , v0 ) + b
(u0 , v0 ) .
u
v
X
X
(u0 , v0 ) and
(u0 , v0 ) are the columns of the Jacobian matrix [DX (u0 , v0 )].
u
v
E(u0 , v0 )
X X
(u0 , v0 )
u u
F (u0 , v0 )
X X
(u0 , v0 ) ,
u v
G(u0 , v0 )
X X
(u0 , v0 )
v v
and the matrix, called the First fundamental Matrix,

"
E(u0 , v0 )
F (u0 , v0 )
F (u0 , v0 )
G(u0 , v0 )
#
.
(6.43)
Then
kx0 (0)k2
= Ea2 + 2F ab + Gb2
"
= (a, b)
E(u0 , v0 )
F (u0 , v0 )
F (u0 , v0 )
G(u0 , v0 )
#
(a, b) .
Thus, the First Fundamental Matrix may be used to compute the length of the three dimensional
tangent vector to a curve x(t) = X(u(t)) in terms of the two dimensional vector u0 (0) = (a, b). Notice
that x0 (0) is a unit vector if and only if Ea2 + 2F ab + Gb2 = 1.
Example 100 (First Fundamental Matrix for a graph). Let f (x, y) be a continuously differentiable
function on R2 . Consider the parameterized surface S given by
X(u, v) = (u, v, f (u, v)) .
X(u, v) =
u

1, 0,
f (u, v)
u

and
X(u, v) =
v

0, 1,
f (u, v)
v

.
266
Therefore,
2

f (u, v)
1+
u

f (u, v)
f (u, v)
=
.
u
u
2

f (u, v)
= 1+
u
E(u, v)
F (u, v)
G(u, v)
For example, for
f (x, y) = x2 + y 2 ,
the First Fundamental Matrix is
"
1 + 4u2
4uv
4uv
1 + 4v 2
Note also that if (u0 , v0 ) is a critical point of f , so that f (u0 , v0 ) = 0, then from (6.51), we
have that
"
E(u0 , v0 )
F (u0 , v0 )
F (u0 , v0 )
G(u0 , v0 )
"
=
#
.
(6.44)
b
Now consider a curve x(t) = X(u(t)). At each time t, we have the unit normal vector N(x(t))
b
and the tangent vector x0 (t) = [DX (u(t)]u0 (t). Since N(x(t))
is a unit vector,

d b
d b
d
b
b
N(x(t)) N(x(t))
=2
1=
N(x(t)) N(x(t))
.
dt
dt
dt

d b
b
Since for each t,
N(x(t)) is orthogonal to N(x(t))
is lies in the tangent plane to the surface at
dt
x(t).
b
As you see from the picture above, the rate of change of N(x(t))
as x(t) moves along the surfaces
0=
b
has to do with the curvature of the surface: If the surface is planar, then N(x(t))
is constant. We
are ready to define the directional curvature at a point x0 on a parameterized surface.
Definition 73 (Directional curvature on a parameterized surface). Let X(u, v) be a twice continuously differentiable parameterized surface in R3 , defined for (u, v) in an open set U R2 . Let x0
be a point on the surface, and let U be a unit tangent vector to the surface at x0 . Let x(t) be any
continuously differentiable curve on the surface with
x(0) = x0
and
x0 (0) = f U .
(6.45)
Then the curvature of the surface in the direction U is the quantity

d b
N(x(t)) x0 (t)
U .
dt
t=0
To minus sign is included so that for a convex surface like the parabolic cup
X(u, v) = (u, v, u2 + v 2 ),
the directional curvatures are all positive (and equal), as we show in the next example.
(6.46)
267
Example 101 (Direct computation of directional curvature). Consider the parameterized surface
given by (6.46). Thus surface is a parabolic up, opening upwards, and has positive curvature in a
obvious intuitive sense of the term.
Let us compute the curvature at x0 = (0, 0, 0) using Definition 73. The tangent plane to the
surface at x0 is the horizontal plane through the origin, so the unit tangent vectors to the surface at
x0 are the vectors of the form
U = (a, b, 0)
with
a2 + b2 = 1 .
Given any such vector, the curve

x(t) = X(ta, t, b) = (ta, tb, t2 )
satisfies (6.45). By the formula computed in Example 99,
b
N(X(u,
v)))(1 + u2 + v 2 )1/2 ( u, v, 1) .
Therefore,
b
N(x(t)))
= (1 + t2 )1/2 ( at, bt, 1)
and so

d b
= ( a, b, 0) = U .
N(x(t)))
dt
t=0
Therefore,

d b
U=UU=1 .
N(x(t)) x0 (t)
dt
t=0
We can obtain a simple formula for the directional curvature as follows: Note that an continuously
b
differentiable curve x(t) in the surface, x0 (t) is tangent to the surface at x(t), while N(x(t))
is
orthogonal to the tangent plant at x(t). Therefore,
b
N(x(t))
x0 (t) = 0
for all t ,
and hence, differentiating at t = 0,

d b
d b
0
b

0=
N(x(t)) x (t)
N(x(t))
x0 (0) + N(x(0))
x00 (0) .
=
dt
dt
t=0
t=0
We conclude that

(6.47)

d b
b
N(x(t))
x0 (0) = x00 (0) N(x(0))
.
dt
t=0
b Now let us see how to compute this

For this reason, we are led to examine the quantity x00 (0) N.
quantity. Suppose that our curve u(t) = (u(t), v(t)) in the parameter plane is twice differentiable,
and define x(t) = X(u(t)) as before.
x00 (0)
(u0 (0))2
+ u00 (0)
2
2X
2X
0
2 X
0
0
(u
,
v
)
+
(v
(0))
(u
,
v
)
+
2u
(0)v
(0)
(u0 , v0 )
0
0
0
0
u2
v 2
uv
X
X
(u0 , v0 ) + v 00 (0)
(u0 , v0 ) .
u
v
268

Since, by construction, both
u0 (0) = (a, b) as above,
X
X
b we have, with
(u0 , v0 ) and
(u0 , v0 ) are orthogonal to N,
u
v
2
2
2
b = a2 X N(u
b 0 , v0 ) + b2 X N(u
b 0 , v0 ) + 2ab X N(u
b 0 , v0 ) .
x00 (0) N
u2
v 2
uv
(6.48)
Therefore , we define numbers L, M and N by

L=
2
b 0 , v0 )
X N(u
u2
M=
2
b 0 , v0 )
X N(u
uv
and then the Second Fundamental Matrix

"
L(u0 , v0 )
M (u0 , v0 )
M (u0 , v0 )
N (u0 , v0 )
and N =
2
b 0 , v0 ) ,
X N(u
v 2
#
.
(6.49)
b
Then if u0 (0) = (a, b), x00 (0) N
b
x00 (0) N
= La2 + 2M ab + N b2
"
= (a, b)
L(u0 , v0 )
M (u0 , v0 )
M (u0 , v0 )
N (u0 , v0 )
#
(a, b) .
Example 102 (Second Fundamental Matrix for a graph). Let f (x, y) be a continuously differentiable
function on R2 . Consider the parameterized surface S given by
X(u, v) = (u, v, f (u, v)) .
2
X(u, v)
u2
2
X(u, v)
v 2
2
X(u, v)
uv

2 f (u, v)
=
0, 0,
u2

2 f (u, v)
=
0, 0,
v 2

2
f (u, v)
=
0, 0,
.
uv

(6.50)
As we have seen in Example 99,

n(u, v) =
f (u, v)
f (u, v)
,
, 1
u
v

.
Therefore
b
N(u,
v) = p
1
1 + kf (u, v)k2
f (u, v)
f (u, v)
,
, 1
u
v

.
Therefore,
L(u, v)
2 f (u, v)
1 + kf (u, v)k2 u2
M (u, v)
2 f (u, v)
.
1 + kf (u, v)k2 uv
N (u, v)
2 f (u, v)
1 + kf (u, v)k2 v 2
(6.51)
269
For example, for

f (x, y) = x2 + y 2 ,
the Second Fundamental Matrix is
2
1 + 4u2 + 4v 2
"
#
.
Note also that if (u0 , v0 ) is a critical point of f , so that f (u0 , v0 ) = 0, then from (6.51), we
have that
"
L(u0 , v0 )
M (u0 , v0 )
M (u0 , v0 )
N (u0 , v0 )
#
= [Hessf (u0 , v0 )] .
(6.52)
The Second Fundamental Matrix enables you to compute the component of the acceleration in a
curve on the surface that is due to the way the surface is curving away from its tangent plane at that
point. As you see, it depends only on the velocity vector of the curve u(t) in the parameterization
plane not at all on u00 (0), the acceleration in the parameter plane, which has nothing to do with the
b is the same for all twice differentiable curves
curvature of the surface per se. In particular, x00 (0) N
X(u(t)) passing through X(u0 , v0 ) that have the same tangent vector
x0 (0) = [DX (u0 , v0 )](a, b) = a
X
X
(u0 , v0 ) + b
(u0 , v0 ) .
u
v
Definition 74 (Curvature). The intrinsic curvature at X(u0 , v0 ) along a curve in S through X(u0 , v0 )
with the tangent vector [DX (u0 , v0 )](a, b) is the quantity
"
S (a, b) := (a, b)
L(u0 , v0 )
M (u0 , v0 )
M (u0 , v0 )
N (u0 , v0 )
#
(a, b) .
We now ask: Among all (a, b) such that the the tangent vector [DX (u0 , v0 )](a, b) is a unit vector,
which give the largest and smallest values of the curvature S (a, b)?
Definition 75 (Principal curvatures and Gaussian curvature). The principal curvatures of S at x0
are the quantities
1 = max{S (a, b) : Ea2 + 2F ab + Gb2 = 1} ,
and
2 = min{S (a, b) : Ea2 + 2F ab + Gb2 = 1} .
The Gaussian curvature K of S at x0 is the product 1 2 .
b
Notice that while 1 and 2 will change sign if one changes the orientation; i.e., the sign of N.
However, the Gaussian curvature does not: It is independent of the choice of an orientation, and is
an intrinsic property of the surface itself.
Notice also that computing
1 = max{ La2 + 2M ab + N b2 : E2a2 + 2F ab + Gb2 = 1 }
270
is a constrained optimization problem: We seek to maximize

f (a, b) = La2 + 2M ab + N b2
subject to the constraint g(a, b) = 1 where g(a, b) = E2a2 + 2F ab + Gb2 . Likewise, computing 2
is a constrained minimization problem. We know how to solve such problems using the method of
Lagrange multipliers.
Theorem 69 (Curvature of Parameterized surfaces). Let S be a twice continuously differentiable
parameterized surface. Then at any pont X(u0 , v0 ) in S, the principal curvatures 1 and 2 are the
two roots of the quadratic equation p(t) = 0 where
p(t) = (L tE)(N tG) (M tF )2 .
(6.53)
Moreover, the Gaussian curvature K is given by

K = 1 2 =
LN M 2
.
EG F 2
(6.54)
All quantities are evaluated at (u0 , v0 ).

Example 103 (Curvature of a paraboloid). Let f (x, y) = x2 + y 2 . Consider the parameterized
surface S given by
X(u, v) = (u, v, f (u, v)) = (u, v, u2 + v 2 ) .
As we have seen above, the First Fundamental Matrix is
"
1 + 4u2
4uv
4uv
#
,
1 + 4v 2
and the Second Fundamental Matrix is

2
1 + 4u2 + 4v 2
"
#
.
Thus the Gaussian curvature K(u, v) is given by

K(u, v) =
(1 + 4u2 + 4v 2 )2
.
2
Next, let us compute the principal curvatures

"
#at (0, 0), which is the unique critical point
" of f . At
#
1 0
1 0
(0, 0), the Frist Fundamental Matrix is
and the Second Fundamental Matrix is
.
0 1
0 1
Then p(t) = (2 t)2 , and so
+ (0, 0) = (0, 0) = 2 .
Proof of Theorem 69: As we have explained above, computing the principal curvatures is a matter
of maximizing and minimizing f (a, b) = La2 + 2M ab + N b2 subject to the constraint g(a, b) = 1
where g(a, b) = E2a2 + 2F ab + Gb2 . Note that the variables here are a and b. Lagranges equation
f (a, b) = g(a, b) works out to
"
L
"
(a, b) =
#
(a, b) .
(6.55)

This means that
"
271
L E
M F
M F
L N
#
(a, b) = (0, 0) .
(6.56)
Since
E2a2 + 2F ab + Gb2 = 1 ,
(6.57)
"
#
L E M F
(a, b) is not the zero vector, and so the matrix
is not invertible, and hence
M F L N
its determinant is zero. Therefore,
(L E)(N G) (M F )2 = 0 ,
and hence that the Lagrange multiplier must be one of the two roots of the quadratic equation
(L tE)(N tG) (M tF )2 = 0 .
We now show that these roots are the principal curvatures.
Let (a1 , b1 ) be a maximizer for our constrained optimization problem. Let 1 be the root of
(L tE)(N tG) = (M tF )2 that is the Lagrange multiplier for which (6.55) is valid at the
maximizer. Taking the dot product of both sides of
#
"
"
E
L M
(a1 , b1 ) = 1
F
M L
(a1 , b1 )
with (a1 , b1 ), and remembering the constraint (6.57), we find

La21 + 2M a1 b1 + N b21 = 1 .
Since (a1 , b1 ) is the maximizer, and 1 is the maximum value, we conclude that 1 = 1 . In the
exact same way, we conclude that 2 = 2 .
At this point, we know that the two principal curvatures are the two roots of the quadratic
polynomial
p(t) = (L tE)(N tG) = (M tF )2 .
(6.58)
Moreover, every quadratic polynomial with roots 1 and 2 can be written in the form
p(t) = C(t 1 )(t 2 ) .
(6.59)
Expanding the two expressions (6.58) and (6.59), and equating coefficients of like powers of t, we
deduce the formula (6.54).
Example 104 (Principle curvatures for a graph at a critical point). Let f (x, y) be a continuously
differentiable function on R2 . Consider the parameterized surface S given by
X(u, v) = (u, v, f (u, v)) .
As we have seen in (6.51) and (6.52), when (u0 , v0 ) is a critical point of f , the First Fundamental Matrix at (u0 , v0 ) is the identity matrix, and the Second Fundamental Matrix at (u0 , v0 ) is
Hessf (u0 , v0 )]. In this case, the polynomial p(t) defined in (6.58) is simply the characteristic polynomial of Hessf (u0 , v0 )], and hence the principle curvatures are exactly the eigenvalues of Hessf (u0 , v0 )].
Thus our new definition of principle curvatures for parameterized surfaces agrees with our old definition for surfaces that are graphs.
272
6.3
Exercises
6.1: Let f (x, y) = x2 y + xy 2 xy. Evaluate the Hessian matrix of f at x0 := (1, 1) and v := (1, 2),
and compute the second directional derivative

d2
.
f (x0 + tv0 )
2
dt
t=0
6.2: Let f (x, y) = x4 + y 4 4xy. Evaluate the Hessian matrix of f at x0 := (0, 1) and v := (1, 1),
and compute the second directional derivative

d2

.
f
(x
+
tv
)
0
0
2
dt
t=0
6.3: Let f (x, y, z) = x2 yz + xy 2 xz. Evaluate the Hessian matrix of f at x0 := (1, 1, 1) and
v := (1, 0, 1), and compute the second directional derivative

d2
.
f (x0 + tv0 )
dt2
t=0
6.4: Let f (x, y, z) = xyz xy + xz. Evaluate the Hessian matrix of f at x0 := (1, 0, 1) and
v := (1, 2, 1), and compute the second directional derivative

d2
.
f (x0 + tv0 )
2
dt
t=0
"
6.5: Let A =
#
. Find the eigenvalues of A and an orthonormal basis of R2 consisting of
eigenvectors of A
6.6: Let f : R2 R be defined by
2
2
xy x y
2
2
x +y
f (x, y) =
(x, y) 6= (0, 0)
(x, y) = (0, 0) .
(a) Show that
4
2 2
4
y(x + 4x y y )
2
2
2
(x + y )
f (x, y) =
x
0
(x, y) 6= (0, 0)
(x, y) = (0, 0) ,
and
4
2 2
4
x(x 4x y y )
(x2 + y 2 )2
f (x, y) =
y
0
(x, y) 6= (0, 0)
(x, y) = (0, 0) .
and that both of these partial derivatives are continuous everywhere on R2 .

(b) Show that for (x, y) 6= (0, 0),
6.3. EXERCISES
273

x6 + 9x4 y 2 9x2 y 4 y 6
.
f (x, y) =
f (x, y) =
x y
y x
(x2 + y 2 )3
Show that the function on the right hand side does not have a limiting value at at (0, 0); i.e.,
x6 + 9x4 y 2 9x2 y 4 y 6
(x2 + y 2 )3
(x,y)(0,0)
lim
does not exist.

f (0, 0) and
f (0, 0) both exist, and compute the values. How does this
x y
y x
result fit with Clairauts Theorem?
"
#
4 2
6.7: Let A =
2 4
eigenvectors of A
"
#
1 2
6.8: Let A =
2 5
eigenvectors of A
#
"
1 2
6.9: Let A =
2 4
eigenvectors of A
(c) Show that
6.10: Let f (x, y) = x2 y + xy 2 xy.

(a) Find all of the critical points of f . Evaluate the Hessian matrix of f at each of these critical
points, and determine where each is a local maximum, a local minimum, a saddle, or undecidable
from the Hessian.
(b) There is one critical point in the interior of the upper right quadrant. Sketch a contour plot of
f in the vicinity of this critical point. Show the computations that lead to the plot.
9
1
6.11: Let f (x, y) = 3xy 3 + 2x + x4 + y 2 .
4
2
from the Hessian.
(b) Choose one of the critical points and sketch a contour plot of f in the vicinity of this critical
point. Show the computations that lead to the plots.
6.12: Let f (x, y) = 3x3 + 5xy + 5x2 5y 2 .
from the Hessian.
(b) Sketch a contour plot of f in the vicinity of each of the critical points. Show the computations
that lead to the plots.
6.13: Let f (x, y) = x2 + y 2 2yx2 .
274
from the Hessian.
(b) Sketch a contour plot of f in the vicinity of each of the critical points. Show the computations
that lead to the plots.
6.14: Let f : R2 R be given by f (x, y) = 4xy x4 y 4 .

d
2 2
3
(a) Let x(t) b given by x(t) = (t + t , t + t ). Compute f (x(t)) .
dt
t=1
(b) Find all of the critical points of f , and find the value of f at each of the critical points.
(c) Compute the Hessian of f at each critical point and determine whether each critical point is a
local minimum, a local maximum, a saddle point, or if it cannot be classified through a computation
of the Hessian.
(d) Does f have a maximum value? Explain why or why not. If it does, find all points at which the
value of f is maximal; i.e, find all maximizers.
(e) Does f have a minimum value? Explain why or why not. If it does, find all points at which the
value of f is minimal; i.e, find all minimizers.
(f ) Sketch a contour plot of f near each critical point.
6.15: Let f (x, y) = x4 + y 4 2x2 y. There is exactly one critical point (x0 , y0 ) with x0 > 0 and
y0 > 0.
(a) Compute the Hessian of f at (x0 , y0 ), and determine whether it is a local minimum, a local
maximum, a saddle point, or if it cannot be classified through a computation of the Hessian.
(b) Let u = (u, v) be a unit vector, and consider the directional second derivative

d2
.
f (x0 + tu, y0 + tv)
dt2
t=0
Which choice of the unit vector (u, v) makes this as large as possible? What is the largest possible
value? Also, which choice of the unit vector (u, v) makes this as small as possible, and what is the
smallest possible value?
(c) Sketch a contour plot of f near (x0 , y0 ).
6.16: Let f (x, y) = x4 + y 4 4xy.
(a) Find all of the critical points of f , and for each of them, determine whether it is a local minimum,
a local maximum, a saddle point, or if it cannot be classified through a computation of the Hessian.
(b) There is one critical point of f in the interior of the upper right quadrant. Let x0 = (x0 , y0 )
denote this critical point. Let u = (u, v) be a unit vector, and consider the directional second
derivative

d2
f (x0 + tu, y0 + tv)
.
2
dt
t=0
Which choices of the unit vector (u, v) makes this as large as possible? What is the largest possible
value? Also, which choices of the unit vector (u, v) makes this as small as possible, and what is the
6.3. EXERCISES
275
(c) Sketch a contour plot of f near (x0 , y0 ).

6.17: Let f (x, y) = 4x2 + y 2 + 4xy (x 1)4 (y 1)4 4x 4y.
(a) The point (0, 0) is a critical point of f . Compute the Hessian of f at this point, and determine
whether it is a local minimum, a local maximum, a saddle point, or if it cannot be classified through
a computation of the Hessian.
(b) Let u = (u, v) be a unit vector, and consider the directional second derivative

d2
.
f (tu, tv)
2
dt
t=0
Which choice of the unit vector (u, v) makes this as large as possible? What is the largest possible
value? Also, which choice of the unit vector (u, v) makes this as small as possible, and what is the
(c) Sketch a contour plot of f near (0, 0).
6.18 Consider the function f (x, y, z) given by
f (x, y, z) = x3 yz 2 + 4xy 3yz .
Determine whether all of the eigenvalues of the Hessian at x0 = (1, 1, 1) are positive or if they are
all negative, or neither.
6.19 Consider the function f (x, y, z) given by
f (x, y, z) = xyz 2 + xy 2 z + x2 yz .
Determine whether all of the eigenvalues of the Hessian at x0 = (1, 1, 1) are positive or if they are
all negative, or neither.
6.20 Compute 1 , 2 and K for all points on the sphere of radius R in R3 .
p
6.21 Compute 1 , 2 and K at all points on the cone z = x2 + y 2 at which z > 0. (The surface is
not differentiable at the apex of the cone.)
6.22 Compute 1 , 2 and K at all points on the surface given by z = xy.
276
Chapter 7
INTEGRATION IN SEVERAL
VARIABLES
7.1
7.1.1
Integration and summation

A look back at integration in one variable
This chapter is focused on the problem of integrating functions of several variables. There are many
ways to think about what it means to integrate a function of one variable, and not all of them
are useful starting points for the transition to several variables. For example, integration is often
thought of as the procedure that undoes differentiation. This is not unreasonable: in practice,
one computes integrals by finding antiderivatives. But suppose we have a function f (x, y) of two
variables. What could it possibly mean to find an antiderivative of f (x, y)? We have come to
understand the gradient as the derivative in two variables, but that is a vector quantity, and so it
would make no sense to seek an antigradient of f (x, y).
Therefore, we begin with a problem in one variable. Our aim is to explain what integrals are from
a point of view that facilitates the transition to several variables. Consider the following problem:
How much work is required to raise a 100 foot flagpole that weighs one pound per foot from horizontal
to vertical?

277
278
CHAPTER 7. INTEGRATION IN SEVERAL VARIABLES
Recall that work is product of force and the distance traveled in moving against the direction of
that force. (Force is a vector quantity, so it has a direction). In the case at hand, the force is gravity.
The direction is straight down, so we are only concerned with vertical displacement.
If we lift a one pound weight one foot, we do one footpound of work. If we lift a one pound weight
ten feet, we do ten footpounds of work. This is all simple multiplication. The flagpole problem,
however, requires calculus because different parts of the flagpole get raised different amounts. Near
the base, there is not much raising going on at all, while the parts of the flagpole near the top are
raised nearly 100 feet. We cannot simply use the formula
work = weight height raised
(7.1)
because there is no one value for height lifted that is valid for the whole flagpole.
The way to carry out the computation using calculus is to first slice the flagpole into small bits
in your mind only; do not ruin the flagpole! Pick a small slice size h > 0, and slice the flagpole
perpendicular to its axis into small blocks that are h feet long. Now raise the flagpole by stacking
the blocks in the right order. We are going to add up the amount of work we do lifting each block
into place to get the total work done lifting the flagpole into place.
The jth block from the base will have to be raised to a height of j h feet. Not everything in
the block gets raised by exactly this amount, but if h is small compared with j h, thw difference
will be small percentage-wise.
Percentage-wise is the key word: The block itself is small, so an approximation of the amount
of work it takes to lift it into place that is off by a factor of 2 would make a small error. But adding
up all of these errors for each of the blocks which will be many in number we could be off by a
factor of 2 for the flagpole as a whole. But if we are only off by a small percentage in each block,
when we add up all of the errors, we will only be off by a small percentage for the flagpole as a whole.
And if we can arrange that the percentage error goes to zero as the size of the small blocks goes to
zero, we can get an exact answer by taking the limit in which the block size goes to zero. This is the
fundamental idea on which the integral calculus is based.
7.1. INTEGRATION AND SUMMATION
279
Returning to our example, up to a small percentage-wise error, we can use the formula (7.1).
Since the flagpole weighs 1 pound per foot, the weight of the block is h pounds.
Hence the work done in raising the jth block is

(h pounds) (jh feet) = (jh2 footpounds) .
Now the key point is that work is an extensive, or in other words, additive, quantity. Hence, letting
N be the total number of blocks, which is 100/h, we have that the total work is
N
X
jh2 .
j=1
Letting xj denote jh, and letting x denote h, this becomes

N
X
xj x ,
j=1
and you recognize this as a Riemann sum for the integral

Z
0
100
100
x2
xdx =
= 5, 000 .
2 0
This is what we get for the sum in the limit as h 0. Hence, the total work done is 5, 000 foot
pounds.
Now let us consider what we have done, and identify the essential steps. Integration means
making whole. This refers to the adding up procedure towards the end of the problem, and we
used an antiderivative namely x2 /2, which is an antiderivative of x to do the sum in the limit
h 0. (This is the passage from the Riemann sum to the integral, with which you are familiar from
single variable calculus).
However, before you can make something whole, you have to first take it apart, and higher
dimensional integration problems generally begin as disintegration problems. Depending on how
280
you choose to slice your problem into bits at the beginning, you can be faced with integration
problems of quite different degrees of difficulty.
So although we are studying integration in this part of the course, much of our effort will
be focused on disintegration we want to do this in a thoughtful, careful way that facilitates the
integration steps at the end. We begin with some simple problems in which the most obvious sort of
disintegration works well.
7.1.2
Integrals of functions on R2
Consider a region D in R2 . To be concrete, suppose that D is the closed unit disk in R2 . That is, D
consists of all points (x, y) satisfying
x2 + y 2 1 .
(7.2)
Suppose that we have a sheet of metal lying in this region, and it has a mass density of f (x, y)
mass units per area units. (Grams per square centimeter if you like). What is the total weight of the
sheet of metal?
If the mass density function f (x, y) were constant, we could use the formula
mass = mass density area .
(7.3)
If x and y are measured in centimeters, the area of D is square centimeters, and so if the density
were a uniform 1 gram per square centimeter, the total weight would be grams.
But suppose that the disk of metal is thinner near the center, and has the mass density
f (x, y) = x2 + y 2 .
What would be the total weight in this case? Less, clearly, but how much less?
The way forward is to disintegrate the the disk into small bits in which the mass density is
effectively constant, and then to apply the formula (7.3) to each of these. This gives us the mass of
each of the pieces. Since the mass of the whole is the sum of the mass of the parts, all we need do is
to add up all of these masses, and make the disk whole again. This is the integration phase.
To disintegrate the disk, we chop it up on a rectangular grid. Let x be the spacing between
the vertical grid lines and let y be the spacing between the horizontal grid lines. Most of the disk
is covered by rectangular tiles of area xy. There are some tiles with more complicated shapes
around the boundary, but these will account for a small percentage of the disk if both x and y
are very small. Hence, let us ignore these for now, and focus on the rectangular tiles. In each of
these, the mass density does not vary much at least when both x and y are very small so it
makes sense to talk about the value of the mass density in the little tile. For each such tile, we have
that the mass is
(mass density in the tile) xy .
Hence the total mass is
X
little tiles
(mass density in the tile) xy .
(7.4)
281
Now we are ready for the integration phase. We can add up the terms in the sum in any order
we like addition is commutative, and the sum is finite. There are two very natural ways to proceed:
We and add up the contributions from the tiles in each column, and then we can add up the sums
for each column, or we can add up the contributions from the tiles in each row, and then we can add
up the sums for each row.
Lets first add up the contributions from the tiles in each column. Suppose that there are M
columns, labeled by j = 1, 2, . . . , M .
Then we have
X
(mass density in the tile) xy
little tiles
N
X
(mass density in the tile) xy
i=1
N
X
little tiles in column j
(mass density in the tile) y x
i=1
(7.5)
If xj is the x coordinate of, say, the middle of the jth column, then the inner sum,
X
(mass density in the tile) y
is the Reimann sum for the integral

Z
b(xj )
f (xj , y)dy ,
a(xj )
where a(xj ) is the y coordinate at the bottom of the jth column and b(xj ) is the y coordinate at the
top of the jth column. In the case at hand, from the equation x2 + y 2 = 1 at the boundary of the
region, we have
q
a(xj ) = 1 x2j
and
b(xj ) =
1 x2j
282
and so, in more concrete terms, our integral is

Z
1x2j
f (xj , y)dy .
1x2j
For any fixed value of xj , this is a garden variety definite integral in the single variable y. Doing it,
we get
2
q
3 1xj
2
y
= 2x2j 1 x2j + (1 x2j )3/2 .
x2j y +
3 1x2
3
j
Going back to (7.5), we see that, upon replacing the inner sum by the integral to which it corresponds
(when viewed as a Riemann sum), we have that the total mass is
N
X
2x2j
x2j
i=1

2
2 3/2
+ (1 xj )
x .
3
Since the values of x in the disk range from 1 to 1, this is the Reimann sum for
Z 1
2
(2x2 (1 x2 )1/2 + (1 x2 )3/2 )dx .
3
1
Using the trigonometric substitution x = sin , this is easily evaluated, and the answer is /2.
In the limit as x and y both tend to zero, the approximations that we made in replacing
sums by integrals, and choosing values in the small tiles, etc, all become increasingly negligible, and
so this is the exact value for the total mass.
This problem makes for a good case study of the process of disintegration and integration. Here
is the general process: Let D be some region given by inequalities of the form
a(x) y b(y)
cxd.
Let f be a continuous function defined on D. We summarize and generalize:

Definition 76 (Two dimensional area integral). Given a continuous function f defined on a set D
R2 that is closed, bounded and has a non-empty interior whose boundary is a piecewise differentiable
R
curve, we define the area integral D f (x, y)dA to be
!
Z
X
f (x, y)dA =
lim
(value of f in the tile) (area of tile) ,
(7.6)
D
max tile diameter0
little tiles
where the limit is taken along a sequence of disintegration of D into tiles in which the maximum tile
diameter in the nth disintegration goes to zero as n goes to infinity. More precisely, we require that
for some sequence {rn } of positive numbers with limn rn = 0, each tile in the nth disintegration
lies inside some disk of radius rn .
Though we will not prove it here, it may be shown that under the conditions on f and D, the
limit exists and is independent of the way one disintegrates D into finitely many tiles, as long as
each of the tiles satisfies the same conditions we have imposed on D itself. We shall only make the
following remarks about the conditions imposed on D and the disintegration.
283
(1) The point of having a piecewise differentiable boundary is that this condition ensures that when
h is very small, an overwhelming percentage of the tiles lie in the interior of D.
(2) The point about the assumption on the diameters of the tiles is this: Suppose for simplicity
that our integrand f is continuously differentiable on R2 ; (This will be the case in almost all of the
problems we consider here.) Since D is closed and bounded, and since kf (x)k is continuous, there
is an x0 inD so that
M := kf (x0 )k kf (x)k
for all
xD.
Then, if x and y are any two points in the same tile of diameter r, kx yk r, and so, by the
Fundamental Theorem of Calculus, The Chain Rule, and the Cauchy-Schwarz inequality
Z

|f (x) f (y)| =

f (x + t(y x)) (y x)dt
Z
kf (x + t(y x))kky xkdt M r .

0
Thus, for any > 0, there is an r > 0 so that if all tiles in a disintegration have a diameter no greater
than r, then the diffenrence between the values of f at any two points in any tile is no greater than
. This is what allows us to regard f as constant on the tiles; in the limit this is exactly correct.
7.1.3
Computing area integrals
We now have a definition of area integrals. The definition does not restrict us to rectangular tiles, but
as our first oder of business, let us systematize our approach to using such tiles. Such an approach
is often efficient when the region D is bounded above by a curve y = b(x) and below by a curve
y = a(x), and lies between the vertical lines x = c and x = d. The following diagram shows D, and
the tiles in the column above x.
Suppose that, as in this diagram, every vertical line slices D in a single line segment (or misses
it altogether). That is, the vertical line through x intersects D in an interval [a(x), b(x)] of y values,
or else is empty. If D were more complicated, the intersection could consist of several intervals, or
284
worse. But for now, let us consider this nice case. Then, using a rectangular disintegration, as above,
R
and summing over columns first, we are led to the following formula for D f (x, y)dxdy:
!
Z
Z
Z
b(x)
f (x, y)dy dx .
f (x, y)dxdy =
(7.7)
a(x)
In the inner integral, x is just a parameter, not a variable, so that this integral is a garden variety
integral in the single variable y. Once it is done, y is eliminated, and what remains is a a garden
variety integral in the single variable x. Do that, and you are done.
Example 105 (Computation of an area integral). Let D be the region bounded above by the parabola
R
y = 1x2 , and below by the parabola y = x2 1. Let f (x, y) = x2 +2xy. Let us compute D f (x, y)dA.
The very first thing to do in such a problem is to make a sketch of D. Translating the verbal
information above into a system of inequalities, we have
x2 1 y 1 x2 .
Here is a plot of the bounding parabolas, which are y = x2 1 and y = 1 x2 :
The region D is the region between the two parabolas. Notice that every vertical line intersects
D in a single segment or else the empty set, and so we can use (7.7) and the disintegration of D into
little rectangular blocks. We only need to determine c and d, and a(x) and b(x).
Since the two parabolas meet at x = 1, so c = 1 is the smallest x value in D, and d = 1 is
the largest x value in D. The upper part of the boundary is y = 1 x2 , so we take a(x) = 1 y 2 .
The lower part of the boundary is y = x2 1, so we take b(x) = x2 1.
Hence, (7.7) becomes
Z
1x2
f (x, y)dxdy =
(x + 2xy)dy dx .
(7.8)
x2 1
!
2
In the inner integral, x is a parameter, and y is the variable of integration. So we treat x as a

constant and have
Z
1x2
x2 1
1x2

(x + 2xy)dy = (x y + xy )
= 2x2 (1 x2 ) .
2
x2 1
285
Now (7.8) reduces to

Z
2x2 (1 x2 )dx =
f (x, y)dA =
1
8
.
15
We get another integration formula by summing over rows first instead of columns. Doing the
sum in (7.6) by summing over rows first amounts to interchanging the roles of x and y so that we
have the alternate formula
Z
f (x, y)dx dy ,
f (x, y)dxdy =
D
b(y)
(7.9)
a(y)
provided each horizontal line intersects D in a single line segment (or not at all). This time, c is
the smallest y value in D, and d is the largest y value in D, and for values of y in between, the
intersection of D with the horizontal line through y is the line segment corresponding to the interval
[a(y), b(y)] of x values.
In the inner integral, y is just a parameter, not a variable, so that this integral is a garden variety
integral in the single variable x. Once it is done, x is eliminated, and what remains is a a garden
variety integral in the single variable y. Do that, and you are done.
Example 106 (Alternate computation of an area integral). Let D be the region bounded above by
the parabola y = x2 1, and below by the parabola y = x2 1. Let f (x, y) = x2 + 2xy. Lets
R
compute D f (x, y)dxdy, but this time by integrating first in x. We can do this using (7.9) since
every horizontal line intersects D in a single segment. We only need to determine c and d, and a(y)
and b(y).
For values of y with 0 y 1, the interval is given by the equation for the upper parabola, and
for values of y with 1 y 0, the interval is given by the equation for the lower parabola. Hence
we break the region into two pieces, the upper region Du and the lower region D` . It is clear from the
definition that
f (x, y)dA =
D
f (x, y)dA +
Du
f (x, y)dA ,
D`
so we just need to compute these separately.

In D` , the endpoints of the segment obtained by slicing the region horizontally at height y are
given by the equation y = x2 1. Solving for x, we find x = 1 + y. Hence in D` we have

p
p
1+y x 1+y .
Therefore, we take a(y) = 1 + y and b(y) = 1 + y, and clearly c = 1 and d = 0. Then (7.9)
gives us
Z
f (x, y)dA =
1+y
Du
1+y
!
(x2 + 2xy)dx dy .
Doing the inner integral, treating y as constant,

3
1+y
Z 1+y

x
2
2
2
(x + 2xy)dx =
+ x y
= (1 + y)3/2 .
3
3
1+y
1+y
Hence
Z
f (x, y)dA =
Du
2
4
(1 + y)3/2 dy =
.
3
15
286
For the upper region, the endpoints of the segment obtained by slicing the region horizontally at
height y are given by the equation y = 1 x2 . solving for x, we find x = 1 y. Hence in Du we

have
p
p
1y x 1y .
Hence we take a(y) = 1 y and b(y) = 1 y, and clearly c = 0 and d = 1. Then (7.9) gives us
Z
f (x, y)dA =
1y
Du
1y
(x + 2xy)dx dy .
Doing the inner integral, treating y as constant,
1y
1y
(x + 2xy)dx =
1y

x3
2
2
+ x y
= (1 y)3/2 .
3
3
1y
Hence
Z
f (x, y)dA =
Du
4
2
(1 y)3/2 dy =
.
3
15
Finally, we have
Z
f (x, y)dA =
D
4
4
8
+
=
,
15 15
15
which is what we found before.

We get the same value both ways of course but notice that the first way was easier.
How much calculation one has to in order to evaluate an integral depends very much on how one
goes about the the disintegration and integration processes.
Both disintegration and integration involve choices how do we slice? Do we add up columns
first, or rows? So far we have only discussed slicing the region D into rectangles, but there are
many other choices to consider. And as we have seen, the order in which we choose to integrate the
variables will affect the amount of work we must do.
7.1.4
Polar coordinates
How would you cut a cake? That would probably depend on the shape of the cake. If the cake were
rectangular, cutting it into square or rectangular slices would seem sensible. But if it were round,
you would probably cut it into wedges. Making cuts along the radii, it is easy to divide a round cake
into, say, a dozen equal pieces. This is not so easy if you only make cuts parallel to the lines in a
rectangular grid.
When we are disintegrating a region D in R2 , it can be quite advantageous, for some of the same
reasons, to slice using a grid of radii and concentric circles:
The basic formula that defines the integral is

!
Z
X
f (x, y)dxdy =
lim
(value of f in the tile) (area of tile) .
D
tile diameter0
287
(7.10)
little tiles
We can use this with tiles of any shape that we find convenient, provided the maximum tile diameter
goes to zero in the limit. Of course, the sine qua non of convenience is that we have a simple formula
for the area of the tiles. This is one of the things that is so attractive about rectangular tiles: The
area of a rectangular tile of width x and height y is simply xy.
Now consider a keystone shaped tile that comes from a wedge of angle , and lies between
the radii r and r + r. What is its area?
The keystone shaped tile can be thought of as the part of the circular wedge with opening angle
and radius r +r, that lies outside the circular wedge of the same angle and radius r. Subtracting
the smaller wedge area from the larger, we are left with the area of the tile.
A circular wedge of opening angle and radius R is the fraction

of a disk of radius R. (That
2
is how we measure angles by the fraction of the circumference they subtend). The area of the disk
is R2 , and hence the area of the wedge is
R2
R2 =
.
2
2
The area of our keystone is therefore the difference of the area of two wedges:
(r + r)2
r2
(r)2
= rr +
.
2
2
2
When both r and are very small, the second term on the right is negligible compared to the
first, and hence:
area of keystone tile rr .
As r and diminish, the error in this approximation diminishes in the sense that it becomes a
negligibly small percentage-wise compared to the main term, rr.
288

Thus, the formula for the area element in polar coordinates is
dA = rdrd .
This is the area of an infinitesimal keystone tile.

The grid that we are using to cut the plane into keystone shaped tiles is based on the polar
coordinate system, and we will need to be able to convert between polar coordinates r and
and Cartesian coordinates x and y to use this slicing strategy. As you see, the keystone tiles are
naturally indexed by r and . Therefore, it is natural to express the integrand f (x, y) in these terms.
This is easy: If we measure counterclockwise from the positive x axis, and if r is the distance
from the origin, then
x = r cos
and
y = r sin .
(7.11)
In particular,
x2 + y 2 = r 2 .
(7.12)
By definition, r is always positive. It has a geometric meaning distance from the origin and
distances cannot be negative. (You may have worked with polar coordinates before using a different
convention in which a negative value of r meant that the point would lie at distance |r| from the
origin in the opposite direction, namely, the one corresponding to + . There are some advantages
to this convention. However, there are disadvantages as well, and these are more important here. In
the examples that follow, we shall use the positivity of r several times.)
Think of (7.11) as a dictionary for translating Cartesian coordinates into polar coordinates. You
might think we would be more interested in formulas for r and in terms of x and y. We do have a
formula for r in terms of x and y, namely (7.12), and we could solve (7.11) for , but actually, what
we really need is just (7.11) itself:
To translate a function f from Cartesian into polar terms, define a new function g(r, ) by
g(r, ) = f (r cos , r sin ) .
289
The meaning of this is that if x is any point in R2 , then we can evaluate f at x by substituting
the polar coordinates of x into g.
Example 107 (Translating a function into polar terms). Let f (x, y) = x2 y Then
g(r, ) = (r cos )2 r sin = r3 cos2 sin .
Let us apply our considerations to the problem of evaluating
R
D
f (x, y)dA. If we cut D into
keystone shaped tiles using polar coordinates, and then want to compute
X
(value of f in the tile) (area of tile) ,
little tiles
we must chose an order in which to add up the contributions from each tile. The one that is most
often convenient is to add up all of the contributions from each wedge, and then to add up the
subtotals for each wedge.
The following diagram shows a region D, with the wedge cut through D by the radii at j and
j+1 , where some small has been fixed, and j = j. For example, suppose we choose some
large integer N , and let = 2/N , so that we divide R2 into N wedges with opening angle , so
that j = j/N .
This wedge has been further broken up into keystone tiles by cutting along circular arc of radius
ri where some small value of r has been chosen and ri = ir.
We now organize our summation as follows: For each j, we hold j fixed, and sum up the
contributions from each of the tiles in the jth wedge. That is, we sum over i first, holding j fixed.
Then we add up these subtotals into the grand total by summing on j:
290
(value of f in the tile) (area of tile)
N
X
g(ri , j )ri r
j=1
little tiles
N
X
little tiles in wedge j
g(ri , j )ri r ,
j=1
where ri = ir is the ith value of r used in our grid.

Notice that the inner sum,
X
g(ri , j )ri r
is just the Riemann sum for an integral. If a(j ) is the smallest value of r in D that lies in the jth
wedge, and if b(j ) is the largest value of r in D that lies in the jth wedge, then this is a Riemann
sum for
Z
b(j )
g(r, j )rdr .
(7.13)
a(j )
Here is a diagram showing a() and b().
The diagram also shows the smallest and largest values of for which the ray in direction
intersects the region D. These are denoted c and d. Clearly a() and b() are only defined for
c d.
The value of the integral (7.13) depends on j of course. For c j d, call it G(j ). There are
no keystones to worry about for other values of j , so our sum reduce to
X
(value of f in the tile) (area of tile) =
little tiles
X
j such that cj d
G(j ) ,
(7.14)

Z
291
G()d. Altogether, we have the formula
and this is a Riemann sum for

c
g(r, )rdr d .
f (x, y)dA =
D
b()
a()
Example 108 (An integral in polar coordinates). Let f (x, y) = x2 , and let D be the region bounded
by the circle
(x 1)2 + y 2 = 1 .
Compute
R
D
(7.15)
f (x, y)dA.
The region is a circle, and though it is not centered, we might expect it to have a nice description
in polar coordinates. Let us see. Simplifying, the equation reduces to
x2 + y 2 = 2x .
Using (7.11), this translates into r2 = 2r cos . Since r is strictly positive except at the origin, (7.15)
reduces to
r = 2 cos
(7.16)
This equation is very simple, and will enable us to find simple expressions for a() and b(), and also
c and d. To do this, draw a diagram, and label the boundary of D with the equation that specifies it:
Notice that the formula (7.16) would give a negative value for r in the second and third quadrants,
but has a positive value in the first and fourth quadrants. This tells us that the region D lives in
the first and fourth quadrants, and c = /2 and d = /2. You see this also in the picture, but
drawing a picture is not always so easy. Hence it is important to see how the values of c and d can
be read off of (7.16).
As for a() and b(), draw in a ray at angle , as in the diagram. It enters D at r = 0, and
leave through the boundary with the equation r = 2 cos(). Hence a() = 0, and b() = 2 cos(). That
takes care of the limits. The rest is easy.
292

Translating the integrand using (7.11),
g(r) = (r cos )2 = r2 cos2 .
Therefore,
Z
f (x, y)dA
/2
2 cos()
2
r cos rdr d
=
/2
/2
r dr cos2 d
=
/2
/2
=
/2
24 cos4
4
cos2 d
/2
Z
=
2 cos
cos6 d .
4
/2
The problem is now reduced to a single variable integral with explicit limits, and for our purposes,
the problem is solved. We will regard all such integrals as trivial for the purposes of this course.
It is a non-trivial matter to make the reduction to such a single variable integral, and once you have
done this, computer programs can do the rest, and give you the numerical value, which in this cases
5
. But computer programs cannot make the reduction to the single variable problem.
is
4
Example 109 (Area enclosed by the Bernoulli lemiscate). The Bernoulli lemiscate is the infinity
symbol curve given by
(x2 + y 2 )2 = 2(x2 y 2 ) .
(7.17)
Let us compute the enclosed area, which is

Z
1dA .
D
That is, to get an area, the integrand should just be 1. (Reflect on the definition to make sure this is
clear).
Since the integrand features x2 + y 2 , which will reduce to r2 in polar coordinates, we will translate
(7.17) into polar terms, hoping for something nice. As it stands, (7.17) is pretty awful. It is a
quartic equation, and solving to find either x as a function of y, or y as a function of x, is a daunting
proposition: It can be done, but is big mess. We do not like messes, big or small. Let us try something
else: perhaps polar coordinates avoid a mess; let us try.
Using (7.11), (7.17) becomes r4 = 2r2 (cos2 sin2 ). Then using the double angle formulas,
and dividing through by r2 , (7.17) reduces to
r2 = 2 cos(2) .
(7.18)
This is progress: The variables are separated, and you can also see from (7.18) that the right
hand side is negative unless
/4 /4
or
3/4 5/4 .
7.2. JACOBIANS AND CHANGING VARIABLES OF INTEGRATION IN R2
293
Hence the curve described by (7.18), or equivalently (7.17), lives in these two angular sectors. Here
is a rough sketch:
You could produce such a sketch by evaluating r =
2 cos(2) for a few values of in the range
/4 /4, drawing those points in, and connecting the dots. You do not have to know in
advance that our equation describes the infinity symbol.
Notice that the equation (7.17) only involves x2 and y 2 , so if (x, y) satisfies the equation, so do
the mirror image points
( x, y)
(x, y)
( x, y) .
That is, we can see from the equation that the region is symmetric under reflection about the xaxis
and about the yaxis. This is not so evident form the rough sketch, but that is O.K.; the equations
make it clear.
Because of the symmetry, the area in the first quadrant is exactly one fourth of the total. Hence
we can take c = 0 and d = /4, and remember to multiply by 4 when we have finished integrating.
p
From the diagram, you see that a() = 0 and b() = 2 cos(2), so the integral we need to do is
!
Z
Z
/4
2 cos(2)
1rdr d .
1
Z
The inner integral is trivial, and we are left with
7.2.1
cos(2)d = 1/2. Multiplying by 4, the area

1
is 2.
7.2
/4
Jacobians and changing variables of integration in R2

Letting the boundary of D determine the disintegration strategy
Consider the problem of computing
R
D
f (x, y)dA where f (x, y) = y, and where D is the open rectangle
bounded by the 4 lines

y + 2x = 1
y + 2x = 3
2y x = 2
2y x = 2 .
(7.19)
294
Here is a picture of the region:
To find the limits of integration, we next work out the coordinates of the vertices by solving the
systems of equations for each pair of crossing lines:
If we integrated in x first we would need to break D in the three separate subregions for
3
1
y
5
5
1
y1
5
1y
7
5
since in each of these regions we need a different formula for a(y) or b(y) horizontal segments at
height y begin and end on the same bounding line in only when y stays in one of these ranges.
If we integrated in y first, we could do better: We would only need to break D into the two
separate subregions for
4
8
4
x
5
5
5
since in each of these regions we need a different formula for a(x) or b(x) vertical segments at x
0x
begin and end on the same bounding line only when x stays in one of these ranges.
So, if these were our only choices, certainly we would integrate in y first. However, there is
something better we can do. Instead of disintegrating D using a grid composed of lines parallel to
the axes, lets disintegrate D using a grid of lines paralell to the bounding lines.
To do this, define new variables
u = y + 2x
v = 2y x .
In terms of these variables, the equation for the our lines bounding D reduce to
u=1 u=3
v = 2
v=2.
(7.20)
295
In the u, v plane, the lines bound an open rectangle with sides parallel to the axes, and we can easily
b
divide it up along a rectangular grid. Let us call the rectangle D.
It will be useful to think of these new coordinates as defining a coordinate transformation: Define
u(x, y) = (u(x, y), v(x, y))
where
v(x, y) = 2y x .
u(x, y) = y + 2x
(7.21)
This coordinate transformation is linear: Let J denote the 2 2 matrix

"
#
2 1
J :=
.
1 2
Then
u(x, y) = J(x, y) .
Since det(J) = 5, the matrix J is invertible, and hence this coordinate transformation has the inverse
transformation
x(u, v) = J
1
(u, v) =
5
"
#
(u, v) =
1
(2u v, u + 2v) .
5
(7.22)
y(u, v) =
u + 2v
.
5
(7.23)
That is, x(u, v) = (x(u, v) , y(u, v)) where

x(u, v) =
2u v
5
We will now use this coordinate transformation to transplant a simple and convenient disinteb onto D, which is possible because the cooridnate transformation has been set up as a
gration of D
b
one-to-one map from D onto D.
b is simply a rectangle with sides parallel to the coordinate axes, so that we may
Recall that D
conveniently disintegrate it using a coordinate grid:
The jth vertical line in this grid is the line

u = 1 + ju
(7.24)
296
where u is the horizontal spacing in the grid, and the ith horizontal line in the grid is
v = 2 + iv
(7.25)
where v is the vertical spacing in the grid. (We are ordering the lines left to right and bottom to
top respectively).
Using (7.2.1) to express (7.24) and (7.25) in terms of x and y instead of u and v we get
y + 2x = 1 + ju
(7.26)
2y x = 2 + iv
(7.27)
and
This gives us two sets of parallel, evenly spaced lines in the x, y plane that divide D up into
similar parallelogram shaped tiles.
This grid is our induced disintegration of D it is induced by the simple rectangular disinteb and the coordonate transformation relating D and D.
b
gration of D,
Using the tiles of this induced disintegration, we will compute
!
Z
f (x, y)dA =
D
lim
tile diameter0
(7.28)
little tiles
Each of these tiles in D corresponds to a tile in the u, v plane, and so we can enumerate the tiles in
our induced disintegration of D using an enumeration of the corresponding tiles in our disintegration
of the rectangle 1 u 3, 2 v 2 is the u, v plane. To compute the integral using these
coordinates, we need to answer two questions:
Given a tile with uj u uj + u and vi v vi + v, what is the value of f (x, y) at some
point (x, y) in the corresponding tile in the x, y plane?
Given a tile with uj u uj + u and vi v vi + v, what is the area of the corresponding
tile in the x, y plane?
It is easy to answer the first question, using the inverse coordinate transformation x(u, v) =
b x(uj , vi ) =
(x(u, v) , y(u, v)) given in (7.23). Since (uj , vi ) is in the j, ith tile in the disintegration of D,
297
(x(uj , vi ) , y(uj , vi )) is in the j, ith tile in the induced disintegration of D. Therefore,

f (x(uj , vi ), y(uj , vi ))
is a representative value for f in the j, ith tile of the induced disintegration of D. (And since f is
nearly constant on this small tile it is a good a repreentative as any other.)
We turn to the second question, concerning the area of the tiles. The tiles in the x, y plane are
the images of tiles in the u, v plane under the linear transformation in (7.22). The j, ith tile is the
parallelogram in the x, y plane with vectices
x(uj , vi )
x(uj + u, vi )
= x(uj , vi ) + uJ 1 e1
x(uj , vi + v)
= x(uj , vi ) + vJ 1 e2
x(uj + u, vi + v)
= x(uj , vi ) + uJ 1 e1 + vJ 1 e2 .
(7.29)
We know that for any vectors x0 , a and b in R2 , the area of the parallelogram with vertices st
x0 , x0 + a, x0 + b and x) + a + b is given by det ([a, b]). Therefore, the area of the tile with the
vertices given in (7.29) is
uv det
1

uv
J e1 , J 1 e2 = uv det(J 1 ) =
.
5
Notice that this area is the same for all tiles, independent if j and i.
With our two questions answered, going back to (7.28), we now have

Z
X
1
f (x, y)dA =
lim (g(uj , vi )
uv
u,v0
5
D
i,j
X X1
=
lim
(g(uj , vi )u v .
u,v0
5
i
j
You recognize the Riemann sums for
Z 2 Z
2
(7.30)

1
g(u, v)du dv .
5
In the case at hand, g(u, v) = (2v + u)/5, and so

Z
Z 2 Z 3
1
f (x, y)dA =
(2v + u)du dv
25 2
D
1
u=3 !
Z 2
1
u2
=
2vu +
dv
25 2
2 u=1
Z 2
1
16
=
(4v + 4) dv =
.
25 2
25
What is the lesson to be drawn from this example? It is that:
By using a disintegration scheme that respects the equations defining the boundaries of D, we
were able to avoid breaking up D into subregions that would have to be handled separately, and we
got very simple limits of integration constants in this case.
298

We have encountered a very useful formula in the last example: The formula for the magnifi-
cation factor of a linear transformation.

Theorem 70 (The determinant as a magnification factor). Let A be any 2 2 matrix. Consider the
corresponding linear transformation from R2 to R2 as a linear transformation from the u, v plane to
b be any closed, bounded set in R2 whose boundary is a piecewise differentiable
the x, y plane. Let D
curve, and let D be its image under the linear transformation. That is, let D be the set of points in
b Then D is also a closed and bounded set in R2
the x, y plane of the form A(u, v) with (u, v) D.
whose boundary is a piecewise differentiable curve, and
b .
area(D) = | det(A)|area(D)
In short, the linear transformation corresponding to A magnifies the area of sets nice enough to
have a well-defined area by a factor of | det(A)|. (Note that it may be the case that | det(A)| < 1, in
which case the magnification amounts to shrinking).
The proof of Theorem 70 is essentially a recapitulation of a calculation we have made in the last
example.
b into little square tiles with sides parallel
Proof of Theorem 70: Let us disintegrate the region D
to the u, v coordinate axes, and with side length h. Then the tile with the lower left hand corner at
u has vertices
u,
buh e1 ,
u + he2
and u + he1 + he2 .
The are of this triangle is of course h2 , and the points in the tile are exactly the points of the form
u + se1 + te2
0 s, t h .
The image of this tile under the linear transformation given by A is the set of points of the form
Au + sAe1 + tAe2
0 s, t h ,
since A(u+se1 +te2 ) = Au+sAe1 +tAe2 . Let us write A = [v1 , v2 ] so that Ae1 = v1 and Ae2 = v2 .
Then the transformed tile is the parallelogram with vertices
Au ,
Au + hv1 ,
Au + hv2
and Au + hv1 + hv2 .
The are of this parallelogram is

| det([hv1 , hv2 )]| = |h2 det([v1 , v2 ])| = h2 | det(A)| .
b the corresponding tile in the induced disinThus for each square tile in our disintegration of D,
299
tegration of D has an are that is exactly | det(A)| times as large. Thus

Z
area(D) =
1dA
D
!
X
=
lim
(area of tile in D)
tile diameter0
little tiles
!
=
lim
tile diameter0
b
| det(A)|(area of tile in D)
little tiles
!!
= | det(A)|
lim
tile diameter0
b
| det(A)|(area of tile in D)
little tiles
Z
= | det(A)|
b .
1dA = | det(A)|area(D)
b
D
7.2.2
The change of variables formula for integrals in R2
The strategy developed in the last subsection can be applied even when the boundary of D is not
given by straight lines. There is very little adaptation required if we remember the main idea of the
Differential Calculus: Up close enough, all nice functions are linear for all practical purposes. Let us
consider an example.
Example 110 (The boundary determines the coordinates). Consider the open set D in the upper
right quadrant bounded by
xy = 1
xy = 3
2x = y
x = 2y .
Let us compute the area of D.

The first step is to sketch a plot of the bounding cures:
Two of the bounding curves are arcs of hyperbolas, and the other two are lines. However, notice
that if we introduce
u = xy
and
v = y/x ,
(7.31)
300
we can write the equations for the boundary as

u=1
u=3
v=2
v = 1/2 .
Again, these four lines bound a coordinate rectangle rectangle in the u, v plane.
To proceed,we use use (7.31) to define a nonlinear coordinate transformation u(x, y) by
u(x, y) = (u(x, y), v(x, y)) = (xy, y/x) ,
(7.32)
which is defined on {(x, y) : x 6= 0} R2 , which includes the set D.

b denote
Think of (7.32) as defining a transformation from the x, y plane to the u, v plane. Let D
the open set in the u, v plane given by
1<u<3
and
1/2 < v < 2
(7.33)
b in a one-to-one manner. As in the last subsection, we shall

Then u(x, y) transforms D onto D
need the inverse transformation x(u, v). The inverse transformation transfers the rectangular grid
b into a convenient disintegration of D.
disintegration of D
To obtain the inverse transformation x(u, v), all we have to do is to solve (7.31) for x and y as
functions of u and v. From (7.23), we see that uv = y 2 . Since D is in the upper right quadrant,
p
y > 0, and so y = uv. Next, x2 = u/v, and since x > 0, x = u/v. This gives us
x=
Thus we obtain
u/v
and
y=
uv
(7.34)
x(u, v) = ( u/v, uv) .
Now consider a small tile with uj u uj + u and vi v vi + v in the u, v plane. The

image of this tile under the transformation x(u, v) is a slightly distorted parallelogram in the x, y
plane with vertices at
x(uj , vi )
x(uj + u, vi )
x(uj , vi + v)
x(uj + u, vi + v) .
The distortion will be slight to the extent that u and v are small everything nice looks linear up
close enough.
To compute the area of this parallelogram, we first apply the approximation
x(u) x(u0 ) + [Dx (u0 )](u u0 )
with the basepoint u0 = (uj , vi ), which is the lower left vertex of the tile in the u, v plane. We have:
x(uj , vi )
= x(u0 )
x(uj + u, vi ) x(u0 ) + [Dx (u0 )](u, 0)

x(uj , vi + v) x(u0 ) + [Dx (u0 )](0, v)
x(uj + u, vi v) x(u0 ) + [Dx (u0 )](u, v)
301
In this approximation, the parallelogram is the image of the rectangle with vertices
(0, 0)
(u, 0)
(0, v)
(u, v)
under the linear transformation induced by [Dx (u0 )], and then translated by x(u0 ).
Translation has no affect on area, and by Theorem 70, the linear transformation multiplies the
area of the original rectangle, namely uv, by the factor | det[Dx (u0 )]|. Therefore, using the
notation introduced above:
The image under x of the tile with uj u uj + u and vi v vi + v is a tile in the x, y
plane whose area is
| det[Dx (u0 )]|uv
up to an error that is negligibly small percentage-wise as u and v both go to zero.
Everything is pretty much as it was in our last example, except that now | det[Dx (u)]| is not a
constant. Computing, we find
1
[Dx (u)] =
2
"
u1/2 v 1/2
u1/2 v 3/2
u1/2 v 1/2
u1/2 v 1/2
Therefore,
| det[Dx (u)]| =
1
.
2uv
This gives us a formula for the area of the image of a small tile at u, v, namely
1
uv .
2uv
This is often referred to as the formula for the area element.
In an area computation, our integrand is 1, which requires no translation. However, we can go
ahead and say what we would do if the integrand were some function f (x, y). We would define g(u, v)
by g(u) = f (x(u)). The definition is such that if (x, y) corresponds to (u, v) under the transformation
x, then f (x, y) = g(u, v).
Going back to the basic formula (7.28), we have
!
Z
area of D =
1dA =
D
lim
tile diameter0
1 (area of tile)
(7.35)
little tiles
Using the tiles induced by the transformation f through the regular rectangular grid on the rectangle
(7.33), we get the Reimann sums for
Z
1/2
Z
1

1
du dv .
2uv
The two integrals are now easily worked out with the result that the area is ln(6).
What we have just worked out is a substitution, or change of variables formula for integrals in
two variables.
302

The general picture is this: Suppose that x is an invertible transformation from some open subset
b of R2 to another open subset D of R2 . Think of x as transforming (at least part of) the u, v plane
D
to the x, y plane so that
(x, y) = x (u, v) .
Since the transformation x is invertible, it sets up a onetoone correspondence between points
b
b induces a disintegration of D.
in D and points D so that any disintegration of D
b with its lower
Consider the image of a rectangular tile of width u and height v sitting in D
left corner at (u, v). As we have explained above, the area of the corresponding tile in the induced
disintegration of D is well approximated by
| det[Dx (u, v]|uv .
Now let f be a continuous function on D. Then, for the tiles in the induced disintegration of D,
we have
(value of f in the tile) (area of the tile)
f (x(u, v))| det[Dx (u, v]|uv ,
b
where (u, v) is some point in the chosen tile in the disintegration of D.
Therefore,
Z
f (x(u, v)) | det(Dx (u, v)|dA .
f (x, y)dA =
D
(7.36)
b
D
On the left, dA is the area element in the x, y plane, and on the right dA is the area element in
the u, v plane. To avoid confusion, it is sometimes helpful to use another common notation, and to
write d2 x to denote the area element in the x, y plane, and d2 u to denote the area element in the
u, v plane.
We can use this notation to write
Z
Z
f (x)d2 x =
f (x(u)) | det(Dx (u)|d2 u .
D
(7.37)
b
D
This may be compared to the formula for substitution, or change of variables, in one dimension.
Suppose x(u) is a differentiable function on R. Then if f is any continuous function of one variable,
we have
Z
Z
f (x)dx =
f (x(u))x0 (u)du
(7.38)
a = x(c) and b = x(d).

Notice that the determinant of the Jacobian of x(u) is the higher dimensional replacement for
for the derivative x0 (u) in the one dimensional formula. However, in the one dimensional formula,
there is no absolute value sign. Why is this?
Suppose that c < d as usual, and suppose that x(u) is invertible. Even though x(u) is invertible,
it might be decreasing, so that a = x(c) > x(d) = b. In this case x0 is negative, but we can cancel
this minus sign with the minus sign that comes from swapping the limits on the left. In other words,
if we define
a
= min{x(c) , x(d)}
and
b = max{x(c) , x(d)}

so that a
< b and [
a, b] defined an interval, we could rewrite (7.38) as
Z
Z
f (x)dx =
f (x(u))|x0 (u)|du
[
a,
b]
303
(7.39)
[c,d]
and now we get a formula that looks even more like (7.47).
In writing the simpler formula (7.38), we are taking advantage of the fact that the real numbers
are ordered. There is no natural ordering of the points in a region of R2 , and so there is no natural
analog of switching the limits of integration.
It is important to stress that the formula (7.38) is valid even if x is not a onetoone function, but
not so (7.39), and not so its higher dimensional analog (7.47). For example, if as u sweeps through
[c, d], the interval x(u) sweeps though the interval [
a, b] three times, then you would need a factor of
b
3 on the left in (7.39) for it to be valid. Similar rules counting the number of times the image of D
covers D under x would allow us to consider transformations that are not invertible. Here, we will
only work with invertible transformations; this suffices for the solution of many practical problems.
Now that we have the change of variables formula (7.47), we can put it to work directly, without
explicitly going through considerations of little tiles. That is not to say that the little tiles way
of thinking is expendable in any way. Among other things, it is essential for setting up integrals that
arise in word problems the only way they arise in real life.
Let us close this subsection with some examples of (7.47) in action. We will focus on how one
b
finds x(u) and hence D.
Actually, in practice, one is led first to a formula for u(x), giving u and v as functions of x and
y. Usually, staring at the definition of D, we come up with some definitions of u(x, y) and v(x, y) in
terms of which one can give a simple characterization of the set D. The first order of business then
is to solve this system of equations to find x and y as functions of u and v, or, in other words, to find
x(u).
Example 111 (Using the change of variables formula in R2 ). Let D be the open set in the upper
right quadrant between the curves
x=
1
y2
and
x=
4
y2
and between the curves
Lets compute
R
D
y = x2
and
y = 4x2 .
u = xy 2
and
v = y/x2 ,
(7.40)
1u4
and
1v4.
(7.41)
(x2 + y 2 )d2 x.
If we define
the region is described by
To find x, we solve (7.40) for x and y in terms of u and v. We eliminate x by forming u2 v = y 3 ,

so y = u2/3 v 1/3 . Next, we eliminate y by forming uv 2 = x3 , so that x = u1/3 v 2/3 . This gives us
x (u, v) = (u1/3 v 2/3 , u2/3 v 1/3 ) .
304
b is the rectangle (7.41).

With this definition of x, D
Next, we compute
1
Dx =
3
"
u2/3 v 2/3
2u1/3 v 5/3
2u1/3 v 1/3
u2/3 v 2/3
Therefore,
det(Dx (u)) =
#
.
5 4/3
v
.
9
Next, with f (x, y) = x2 + y 2 ,

f (x(u, v)) = u2/3 v 4/3 + u4/3 v 2/3 .
Hence, from (7.47), we have
Z
Z
5
2
f (x)d x = (u2/3 v 4/3 + u4/3 v 2/3 ) v 4/3 d2 u
9
D
b is jst the rectangle (7.41), this becomes

and since D

Z Z 4
5 4
2/3 8/3
4/3 2/3
(u v
+u v
)du dv .
9 1
1
Integration in R3
7.3
7.3.1
Reduction to iterated integrals in lower dimension
What we have studied so far concerning integrals of functions on R2 readily extends to integrals of
functions on R3 .
The basic formula defining the integral of a continuous real valued function over a region D R3
is
!
f (x, y, z)d3 x =
lim
box diameter0
(value of f in the box) (volume of box)
(7.42)
little boxes
where the sum is over the little boxes in a disintegration of D into a disjoint union of sets, here
called boxes whose volume we know how to measure, or at least accurately estimate. As before,
we require that the maximum diameter of the boxes tends to zero in the limit. This is the obvious
generalization of (7.10).
Then, as in two dimensions, if f is any continuous real valued function on R3 , and D is any
regular bounded subset of R3 , the limit in (7.42) exists no matter what kind of decomposition we use
and no matter where in the box we evaluate f (as long as we can compute the volume of the boxes,
and as long as the maximum diameter of the boxes goes to zero).
The simplest example of such a disintegration is to use the coordinate planes to slice the region
into cubes of side length h > 0. This gives us a particularly easy formula for the volume of the boxes
- each one has volume h3 .
For integers j, k, ` and h > 0, define
xj := jh
yk := kh
and z` = `h .
7.3. INTEGRATION IN R3
305
Then the planes

x = xj , y = yk , z := z` , j, k, ` <
slice up R3 into a regular grid of cubes of side length h. Of course, they also slice up any subset
D R3 into sets we shall call boxes, each of which is contained in one of the cubes. If the set D has
a nice boundary and h is small, so that most of D is at least a distance nh from the boundary,
most of the boxes will be entire cubes. (Note that nh is the diameter of a cube of side length h.)
When we do the sum over all of the little boxes in (7.42), we can do the sum in any order we
like. Here is one good order of summation: For each `, let Dz` be the intersertion of D with the slab
{ (x, y, z) : z` z < z`+1 } .
We then have
X
(value of f in the box) (volume of box) =
little boxes in D
(value of f in the box) (volume of box) . (7.43)
little boxes in Dz`
Let the tile associated to each box be the bottom of the box; i.e., the intersection of the box
with the plane z = z` . Thus there is one tile to each box. Let us evaluate f somewhere in the tile
since we are free to evaluate it anywhere in the box. Then since the area of the tile is h2 and the
volume of the box is h3 ,
(value of f in the box) (volume of box) =
little boxes in Dz`
(7.44)
little tiles in Dz`
It is now easy to recognize
X
lim
h0
little tiles in Dz`
as an area integral in R2 . Specifically, for each z R, define

b z = { (x, y) : (x, y, z) D } .
D
b z is the slice of D by the plane z = z, projected onto the x, y plane. Then we have
Note that D
lim
h0
(value of f in the tile) (area of tile) =
little tiles in Dz`
Z
f (x, y, z` )dxdy .
bz
D
`
(7.45)
306

Now combining (7.43), (7.44) and (7.45), we obtain (formally interchanging limits and sums
without justification, but in a way that can be justified), we obtain
!
lim
h0
(value of f in the box) (volume of box)
little boxes in D
lim
h0
X Z
`
! !
f (x, y, z` )dxdy h
bz
D
`
b z is non-empty, and let b

Now let a be the greatest lower bound on the set of all z such that D
b z is non-empty. Then the right hand side is
be the least upper bound on the set of all z such that D
the limit of Riemann sums for the integral
Z
Z

f (x, y, z)dxdy dz .
bz
D
Thus we finally have:

Z
f (x, y, z)d3 x =
Z

f (x, y, z)dxdy dz
(7.46)
bz
D
b z defined as above.
with a, b and D
This formula reduces the integration of functions over the three dimension set D to an integration
over its two dimensional slices, followed by a one dimensional integral to put everything together.
Since we learned in the previous section how to do the integrals over the two dimensional slices,
and since we know how to do one dimensional integrals, we know how to compute integrals of functions
over three dimension sets D. The derivation of this formula can be completely justified under the
assumptions that f is continuous, and D is bounded has a reasonably smooth boundary.
We shall return later to a more careful justification of this formula. In the rest of this section,
we focus on explaining how it is used.
First, it is not necessary to integrate in z last: We can do the sums in any order we like, so
we can integrate in any order we like. And in general, we are likely to like some orders better than
others: How easy or complicated the limits of integration are will depend strongly on the chosen
order of integration.
7.3.2
The change of variables formula for integrals in R3
The same reasoning that led to the change of variables formula in R2 yields a change of variables
b R3 onto D R3 .
formula in R3 : Let x be a differentiable and invertible transformation from D
Then, for any continuous function f on D,
Z
Z
f (x)d2 x =
f (x(u)) | det(Dx (u)|d3 u .
D
b
D
(7.47)
307
The first examples to consider are the mappings x associated to standard coordinate systems on
3
R . For example, consider

x(r, , )
(x(r, , ) , y(r, , ) , z(r, , ) )
(r sin cos , r sin sin , r cos ) ,
(7.48)
where
0
< 2
This is the spherical coordinate system for R3 .

An easy computation shows that kx(r, , )k = r2 , so as r is held fixed, and and vary,
x(r, , ) ranges over the sphere of radius r in R3 .
We now compute
x
r
y
[Dx (r, , )] =
r
z
r

sin cos
y
=
sin sin
z
cos
r sin sin
r sin cos
0
r cos cos
r cos sin .
r sin
Taking the determinant of the right hand side, we find

det([Dx (r, , )])
r2 sin [(sin2 + cos2 )(sin2 + cos2 )]
r2 sin .
(7.49)
Since sin is non-negaitive for 0 , | det([Dx (r, , )])|dd = sin dd, and hence the
volume element dV for spherical coordinates is given by
dV = sin dd .
(7.50)
Example 112 (The average height in the upper hemisphere). Let V be the upper hemisphere of
radius R. That is, V consists of the points (x, y, z) satisfying
x2 + y 2 + z 2
z
R2
0
The average height in V is

R
zdV
RV
.
1dV
V
The denominator is the total volume of V. Thus, from (7.42), this ratio is

!
X
volume of box
lim
(value of z in the box)
.
box diameter0
volume of V
little boxes
308
That is, the ratio of the integrals is the weighted average of the heights of all the little boxes in a
disintegration of V into infinitesimal boxes, where the weighting is by the fractional volume of the
box.
4 3
R , so we only need to compute the
3
integral in the numerator (though it would be a good exercise to go back and compute the integral in
We already know that volume of the sphere of radius R is
the denominator afterwards, using the same method).

Translating the description of V into spherical coordinates using (7.48) we find r2 R2 . since
both r and R are non-negative, this means r R. Likewise, z 0 becomes r cos 0, which, since
r > 0 except at the origin, means /2. Then going back to (7.50) we have, for this problem,
0
r R
< 2
/2 .
As r, and range over the sets described by (7.51), x(r, , ) ranges over V, the upper hemisphere of radius R.
Z
To compute
zdV we first translate z into spherical terms. By (7.48), z = r cos , and then
V
from (7.50) and (7.51)

Z
Z
zdV
/2
r cos sin dr d d
0
Z
=
Z
/2
1d
0
! Z
cos sin d
r3 dr
(2)(1/2)(R4 /4) =
4
R .
4
(7.51)
Thus, the average height, weighted by volume, in the upper hemisphere of radius R is

1
2 3
4
3
R
R = R.
3
4
8
This makes sense: There is more volume in the lower part of V than the upper part, so lower values
of z will get a higher weight than higher values of z. Therefore, we certainly expect an answer that
is less the R/2. If anything, the surprise is that the actual value is as close as it is to R/2.
The next example to consider is the change of variables associated to cylindrical coordinates:
Let consider
x(r, , z)
= (x(r, , z) , y(r, , z) , z(r, , z )

= (r cos , r sin , z) ,
where
0
< 2
<
z < .
(7.52)
309
An easy computation shows that kx(r, , )k = r2 + z 2 , so as r is held fixed, and and z vary,
x(r, , z) ranges over the cylinder of radius r in centered on the z-axis in R3 .
We now compute
x
r
y
[Dx (r, , z)] =
r
z
r
cos
z
= sin
z
0
z
r sin
r cos
0
0 .
Taking the determinant of the right hand side, we find

det([Dx (r, , z)]) = r(sin2 + cos2 )] = r .
Hence the volume element dV for cylindrical coordinates is given by
dV = rdrddz .
(7.53)
Example 113 (Total mass of an object). Consider an object that has a mass density of %(x, y, z) =
x2 + y 2 grams per cubic centimeter at (x, y, z), and that occupies the region V given by
(y 1)2 + x2
x2 + y 2
Let us compute the total mass of the object. This is given by the integral
Z
%(x)dV .
V
We will compute this in cylindrical coordinates. Let us first translate the description of V into
cylindrical terms. We find (y 1)2 +x2 1 = x2 +y 2 2y = r2 2r sin , so from (y 1)2 +x2 1 0,
we obtain
r2 2r sin 0 .
Away from the z-axis (which has zero volume), r > 0, and so we conclude that
0 r sin .
Since r 0, sin 0, and so 0 .
Next, z 2 x2 + y 2 becomes |z| r, and so, altogether, we have
0
r sin
z r.
Next, translating %(x) into cylindrical terms, we find

%(x) = r2 .
310
Thus,
Z
Z
%(x)dV
sin
r dz dr d
r
sin
3
2r dr d
=
0
Z
=
0
7.4.1
7.4
Z
1
3
sin4 d =
.
2
32
Integration on parameterized surfaces

Parameterized surfaces
A parameterized surface in R3 is a continuously differentiable function x(u, v) defined on an subset

U of R2 with values in R3 such that [Dx (u, v)] has rank 2 at each (u, v) U . For example consider
U = { (u, v) : 0 < u < , 0 < v < 2 } ,
(7.54)
x(u, v) = ( sin u cos v, sin u sin v, cos u) .
(7.55)
and
Computing kx(u, v)k we find

kx(u, v)k =
sin2 u(cos2 v + sin2 v) + cos2 u = 1 .
Thus, for each (u, v) U , x(u, v) lies in the unit sphere, and x(u, v) is a parameterization of
the unit sphere in R3 , take away the semicircle running between the North and South Poles along
( sin u, 0, cos u) for 0 u . We exclude this semicircle to keep U open, and to keep x(u, v) one-toone. The price we pay is that this is not a parameterization of the entire unit sphere. However, what
we have left out accounts for none of the surface area of the unit sphere. For the purposes of this
section, the fact that we have parameterized a part of the sphere that accounts for all of its surface
area, and have done so in a one-to-one and continuously differentiable way, will be what matters.
Other examples of parameterized surfaces are given by the graphs of real valued functions f
defined on R2 . Given such a function, define
x(u, v) = (u, v, f (u, v)) .
A surface in R3 can also be given in implicit form. For example, the equation
x2 + y 2 + z 2 = 1
(7.56)
has the unit sphere in R3 as its solution set. Finding a parameterization of the sphere means to
find an explicit description of the solution set of this equation. The parameterization given above is
one way to do this, and is not only a parameterization, but a continuously differentiable one with a
Jacobian of rank 2 for each choice of the parameters.
Finding such parameterizations for implicitly defined surfaces is a key step in the solution of
many problems involving such surfaces. In the next example, we find such a parameterization.
7.4. INTEGRATION ON PARAMETERIZED SURFACES
311
Example 114 (Parameterizing a surface). Consider the equation

(x2 + y 2 + z 2 )2 = 2(z 2 x2 y 2 ) .
To parameterize the corresponding surface, we must solve this equation. To do this, we rewrite the
equation in terms of spherical coordinates:
x = r sin u cos v , y = r sin u sin v , z = r cos u ,
where (u, v) U where U is given by (7.54). In these variables, the equation becomes
r4 = r2 (cos2 u sin2 u) = r2 cos(2u) .
The parameter v has dropped out of the equation, which facilitates its solution.
Since r = 0 corresponds to the point (0, 0, 0), which is one solution, let us assume r 6= 0, and
find all other solutions. Dividing by r2 and taking a square root, we find:
p
r = cos(2u) .
For values of u such that cos(2u) < 0, we have no solution. In fact for r > 0, we must have
cos(2u) > 0. The values of u corresponding to solutions of our equation, other than (0, 0, 0), are
0 < u < /4 and 3/4 < u < .
Now, we are ready to do the parameterization: First write out the general point in R3 using our
chosen system of coordinates.
x(r, u, v) = ( r sin u cos v , r sin u sin v , r cos u ) .
p
Now use the equation r = cos(2u) to eliminate the variable r. The other two variables become the
parameters. We obtain:
x(u, v) =
cos(2u)( sin u cos v , sin u sin v , cos u ) .
This gives us our parameterization. Here is a plot of the surface:
312

This is an exapmle of a surface of revolution; it is the result of rotating the Bernouli Lemiscate
in the x, z plane about the z-axis.

Whenever we need to parameterize a surface, the procedure will always be the same: We find
some system of coordinates in which we can use the equation defining the surface to express one of
the variables in terms of the other two. (In the previous example, we used the equation to express r in
terms of u and v, though the dependence on v turned out to be trivial.) We then use this expression
to eliminate this variable from the expression for a general point in R3 in the chosen coordinate
system. The two remaining variables become the two parameters.
7.4.2
The tangent plane to a differentiable parameterized surface
Let x(u, v) be a parameterized surface defined on an open set U R2 . The parameterized surface is
differentiable in case x is a differentiable function everywhere on U .
Let u0 = (u0 , v0 ) U . Then the best affine approximation to x(u) at u0 is
x(u) x(u0 ) + [Dx (u0 )](u u0 ) .
(7.57)
Here, [Dx (u0 )] is the 3 2 matrix given by

x(u, v)
y(u, v)
[Dx (u0 )] =
z(u, v)
u
x(u, v)
y(u, v)
.
z(u, v)
v
(7.58)
To work with this more conveniently, let us define the vectors

x(u, v)
y(u, v)
Tu :=
z(u, v)
u
and
x(u, v)
y(u, v)
Tv :=
z(u, v)
v
(7.59)
Thus we can rewrite (7.58) as

[Dx (u0 )] = [Tu (u0 ), Tv (u0 )] ,
(7.60)
and then, introducing relative coodinates u

:= u u0 and v := v v0 , we can rewrite (7.57) as
x(u) x(u0 ) + [Tu (u0 ), Tv (u0 )] (u u0 ) = x(u0 ) + Tu (u0 )
u + Tv (u0 )
v.
(7.61)
The right hand side is a parameterized plane with base point x(u0 ) and direction vectors Tu (u0 )
and Tv (u0 ).
313
Definition 77 (Tangent plane to a parameterized surface). Let x(u, v) be a differentiable parameterized surface defined on an open set U R2 . For u0 U , let Tu (u0 ) and Tv (u0 ) be given by
(7.59). These are the coordinate tangent vectors to the parameterized surface at x(u0 ). The plane
parameterized by
z(
u, v) := x(u0 ) + Tu (u0 )
u + Tv (u0 )
v
is the tangent plane to the parameterized surface at x(u0 ). The parameterized surface is nondegenerate at x(u0 ) in case Tu (u0 ) Tv (u0 ) 6= 0, in which case we define N(u0 ), the unit normal
to the parameterized surface at x(u0 ) by
N(u0 ) :=
1
Tu (u0 ) Tv (u0 ) .
kTu (u0 ) Tv (u0 )k
Example 115 (Computation of a tangent plane). Let us compute the tangent plane to
x(u, v) :=
p
cos(2u)( sin u cos v , sin u sin v , cos u )
at x(u0 ) where u0 = (/6 , /4). We compute

1
((cos(v)(cos(2u) cos u sin(2u) sin u) ,
Tu (u, v) = p
cos(2u)
sin(v)(cos(2u) cos u sin(2u) sin u) , cos(2u) sin u sin(2u) cos u)
and
Tv (u, v) :=
cos(2u)( sin u sin v , cos u cos v , 0) .
Evaluating both vectors at u0 = (/6 , /4), we find
Tu (u0 ) = (0, 0, 2)
and
Tv (u0 ) =
1
( 1, 1, 0) .
4
Evaluating x(u) at u0 = (/6 , /4), we find

x(u0 ) =
1
(1, 1, 6) .
4
Thus, the tangent plane is givne in parametric form by

z(
u, v) =
1
1
(1, 1, 6) + u
(0, 0, 2) + v ( 1, 1, 0) .
4
4
It is also easy to compute an equation for the tangent plane. We already have the base point,
x(u0 ). The obtain a normal vector, we compute
1
2
Tu (u0 ) Tv (u0 ) = (0, 0, 2) ( 1, 1, 0) =
(1, 1, 0) .
4
4
Thus, the vector (1, 1, 0) is normal to the tangent plane, and so
(1, 1, 0) (x x(u0 )) = 0
is an equation for the tangent plane. working this out explicitly, we find
x+y =
1
.
2
314

Here are two views of the tangent plane together with the surface near the basepoint:
Here is a view where we have zoomed in closer:
And finally, here is a view where we have zoomed in closer still:
315
In the last plot, you can see that near the basepoint, the graph of the tangent plane is indistinguishable from the graph of the surface.
7.4.3
The surface area of a parameterized surface
Let x(u, v) be a differentiable parameterized surface defined on an open set U R3 . The mappling
(u, v) 7 x(u, v)
induces a coordinate grid on the parameterized surface, induced by the coordinate grid in the u, v
plane. For instance, if
x(u, v) :=
cos(2u)( sin u cos v , sin u sin v , cos u ) ,
here is a plot of x(u, v)

(u, v) [1/4, 13] [1/4, 1/2] .
316

The grid you see on this piece of the surface consists of lines of constant u and constant v. They
carve the surface up into little tiles. Each of these has vertices of the form
x(u0 , v0 )
x(u0 + h, v0 )
x(u0 , v0 + h)
and x(u0 + h, v0 + h) ,
for some small value of h > 0.

To the extent that the tangent plane approximation is valid, the tile is the parallelogram in R3
with a corner at x(u0 ) and sides hTu (u0 ) and hTv (u0 ) issuing from this corner. The area of the
paralleleogram is given by the magnitude of the cross product of the vectors giving the sides:
surface area of parallelogram = h2 kTu (u0 ) Tv (u0 )k
Now if we add up the surface areas of the parallelograms, we get a good approximation of the
area of this piece of the parameterized surface. Taking the limit h 0, the approximation becomes
exact, and so the area of this piece of the parameterized surface is given by
!
X
lim
h2 (the value of kTu Tv k in the tile .
h0
littletiles
This of course is nothing other than the limit of Riemann sums for the integral
!
Z
Z
1/2
1/3
kTu Tv kdu dv .
1/4
1/4
Integrating over any open U set on which x(u) is defined and differentiable, we obtain in the
same way that the surface area of the part of the surface with parameters in U is given by
Z
surface area =
kTu Tv kd2 u .
(7.62)
Just as we did with arc length, we define the surface area element dS by
dS = kTu Tv kd2 u .
We may then express the integral formula for the surface area of a parameterized surface S as
Z
surface area =
1dS.
(7.63)
S
Notice the difference between (7.62) and (7.63): In (7.62), we consider the domain of integration
to be U , the set of parameter values. Indeed, when it comes to actual computation, this is what we
will be working with. In (7.63), we consider the domain of integration to be the surface S itself, and
we do not refer to any particular set of coordinates. This puts the emphasis on what the integral
actually represents: The sum of all of the contributions of the various area elements to the total area.
We will use either sort of notation, depending on what is best suited to the matter at hand.
Example 116 (Computing surface area). Let S be the part of the paraboloid z = 1 x2 y 2 that
lies above the plane x + z = 1. Let us compute the surface ares of S.
317
The key to doing this is coming up with a good parameterization. To find the intersection of the
plane and paraboloid, we equate their z values and find
1 x2 y 2 = 1 x
which is the same as
x2 + y 2 = x
(x 1/2)2 + y 2 = 1/4 .
or
This is the circle bounding the disk in the x, y plane centered on (1/2, 0) with radius 1/2. This is
what we would see in a top view diagram. Our surface S is the part of the paraboloid that lies above
this disk. Let us use cylindrical coordinates:
(x, y, z) = (r cos , r sin , z) .
The equation for the paraboloid is x = 1 r2 , and so we have our parameterization
x(r, ) = (r cos , r sin , 1 r2 ) .
The equation x2 + y 2 = x translates to r2 = r cos , so the limits on our parameters are
0 r cos
/2 /2 .
and
We next compute
Tt T = (2r2 cos , 2r2 sin , r) ,
and hence the surface area elelemt dS is given by
p
dS = r 4r2 + 1drd .
Hence the surface area is given by
Z
/2
cos
1dS =
S
/2
!
p
r 4r2 + 1dr d .
The inner integral is easily done by substitution:

Z
cos
r
0
4 cos2 +1
p
1 2 3/2
1
4r2 + 1dr =
u
=
((4 cos2 + 1)3/2 1) .
83
12
1
We finally have
Z
f (x, y, z)dS =
S
1
12
/2
((4 cos2 + 1)3/2 1)d .
/2
This may be evluaated in terms of special functions (incomplete elliptic integrals of the first kind),
and perhaps more meaningfully, one finds the numerical value
1.21458608... .
Example 117 (Surface area again). Let V be the region in R3 that lies inside the sphere x2 +y 2 +z 2 =
p
4, and above the graph of z = 1/ x2 + y 2 . Compute the total surface area of its boundary S. (There
are two pieces to the boundary.)
318

First of all, we must parameterize each of the two pieces of the boundary S. The boundary
equations are simple in cylindrical coordinates. We write

x(r, ) = (r cos , r sin , z) .
We then use z =
(7.64)
4 r2 to eliminate z in (7.65), obtaining

p
x1 (r, ) = (r cos , r sin , 4 r2 ) .
(7.65)
as the parameterization of the upper part of the boundary; let us call this S1 .
We next use z = 1/r to eliminate z in (7.65), obtaining
x2 (r, ) = (r cos , r sin , 1/r) .
(7.66)
as the parameterization of the upper part of the boundary; let us call this S2 .
We now determine the range of the parameters r and in our parameterization.
From r2 + z 2 = 4 and z = 1/r, we deduce
(r2 )2 4(r2 ) = 1 .
This quadratic equation for r2 has the roots r2 = 2
at
q
2
r=
3. Hence the two bounding surfaces intersect

q
and
r=
2+
Hence U is the set of vectors (r, ) with

q
2 3
2 .
2+
3.
Next, we work out the area element dS1 for the upper surface S1 . We compute
2

r cos
r2 sin
Tr T (r, ) =
,
,r .
4 r2
4 r2
We then compute
2
2
4

r cos
r2 sin
r2 (4 r2 )
4r2
= r

,
,
r
+
=
.

2
2
4r
4r
4 r2
4 r2
4 r2
Thus,
dS1 =
2r
drd .
4 r2
We then have
Z
surface area of S1 =
0
Z 2+ 3
Z 2
2r

dr d =
2 2d = 4 2 .
2
4r
2 3
0
For the other part of the boundary, we work out the area element dS2 for the upper surface S2 .
We compute

Tr T (r, ) =
cos sin
,
,r
r
r

.
319
We then compute

2
cos sin

= 1 + r2 .
,
,
r

r
r
r2
Thus,
dS2 =
p
r2 + r2 drd .
We then have
Z
surface area of S2
=
0
2 3
"
=
Z 2+ 3 p

r2 + r2 dr d
1
2 + arctanh
2
!
1
2
1
arctanh
2 31
2
!#
2
1
.
2 3+1
Combining the two computations, we get the total area.

Here is are plots of S1 , S2 :
Here is a plot of S1 S2 :
We may also integrate functions defined on parameterized surface S. For example, let S be the
centered sphere of radius R > 0. Suppose it has a mass density of % grams per centimeter squares.
320
If it is rotating about the z-axis with angular velocity , the total kinetic energy is
Z
% 2
(x + y 2 ) 2 dS ,
S 2
and by definition, the moment of inertia of the spherical shell is the quantity I such that the total
kinetic energy is given by
1 2
I .
2
Since the total mass of the shell is M = 4R2 %, we have
Z
Z
1
I = % (x2 + y 2 )dS = M
(x2 + y 2 )dS .
4R2 S
S
(7.67)
In the next example, we compute the integral on the right.

Example 118 (The moment of inertia of a spherical shell). Let S be the centered sphere of radius
R > 0. Let f (x, y, z) = (x2 + y 2 )/2. Let us compute
Z
f dS .
S
First, we need to parameterize the surface. We could use spherical coordinates, but we can express
the integrand more simply in cylindrical coordinates, and the parameterization is almost as simple in
cylindrical coordinates.
The equation for the sphere in cylindrical coordinates is
r 2 + z 2 = R2 .
Therefore, we eliminate z = R2 r2 , and obtain

x(r, ) = (r cos , r sin ,
R2 r 2 ) .
There are two pieces of the surface, say S+ and S corresponding to the two choices for the sign. By
symmetry, we can integrate over S+ , and double our answer.
The domain U of integration is
0<r<R
and
0 < < 2 .
We then work out

Tr T (r, ) =
r2 cos
r2 sin
,
,r
R2 r 2
R2 r 2

,
and so
dS =
Rr
drd .
R2 r 2
Since f (r cos, r sin, z) = r2 , we have

Z
Z
f dS =
S+

4
Rr
2R
4R4
r
dr d = 2
=
.
3
3
R2 r 2
2
7.5. EXERCISES
321
Remembering to double our answer, we have

Z
8R4
.
f dS =
3
S
The total mass M of the spherical shell is its area times the density %.
Thus, going back to the remarks made right before the example, and (7.67) in particuar, the
computation shows that the moment of inertial I of a spherical shell of radius R and mass M is given
by
I=
7.5
2
M R2 .
3
Exercises
7.1 Let f (x, y) = x3 y, and let D be the region that lies to the right of the parabola x = y 2 , and
below the line 2y = x.
R
(a) Write down D f (x, y)dA in terms of integrated integrals, integrating in x first, then y.
R
(b) Write down D f (x, y)dA as an integrated integral integrating in y first, then x.
(c) Evaluate one of the integrals.
7.2 Let f (x, y) = x2 y 2 , and let D be the region that lies inside both of the circles (x 1)2 + y 2 = 4
and (x + 1)2 + y 2 = 4.
R
(a) Write down D f (x, y)dxdy as an integrated integral, integrating in x first, then y.
R
(b) Write down D f (x, y)dxdy as an integrated integral, integrating in y first, then x.
7.3 Let f (x, y) = x2 y 2 , and let D be the region that lies below the parabola y = 4 (x 2)2 and
above the x axis.
R
(a) Write down D f (x, y)dxdy as an integrated integral, integrating in x first, then y.
R
(b) Write down D f (x, y)dxdy as an integrated integral, integrating in y first, then x.
7.4 Let f (x, y) = xy, and let D be the region bounded by the lines y = x, y = 3x, and y = 5x 6.
R
(a) Write down D f (x, y)dxdy in terms of integrated integrals, integrating in x first, then y.
R
(b) Write down D f (x, y)dxdy in terms of integrated integrals, integrating in y first, then x.
7.5 Let f (x, y) = x2 +y 2 , and let D be the region bounded by the lines y = x, y = x and y = 52x.
R
(a) Write down D f (x, y)dxdy in terms of integrated integrals, integrating in x first, then y.
R
(b) Write down D f (x, y)dxdy in terms of integrated integrals, integrating in y first, then x.
7.6 Let f (x, y) = y, and let D be the region bounded by x + y = 2, x + y = 4, x2 y = 1 and x2 y = 2.
R
Compute D f (x, y)dxdy.
322
7.7 Let D be the region bounded by x4 + y 4 = 1. Compute its area. (Use symmetry to conclude
that the area is 4 times the area of the piece in the upper right quadrant, and set up an integral to
compute that). Leave your answer in the form of an explicit integral over one variable. If you do this
in the way that is intended, you will be left with what is known as an elliptic integral. These are well
studied and Maple, for example, is programmed to deal with them.
7.8 Let f (x, y) = xy, and let D be the region in the positive quadrant of R2 bounded by xy = 1,
R
xy = 2, y/x = 1 and y/x = 2. Compute D f (x, y)dA.
7.9 Let f (x, y) = y, and let D be the region inside both of the circles
(x 1)2 + (y 1)2 = 2
Compute
R
D
and
(x + 1)2 + (y 1)2 = 2 .
f x, y)dA.
7.10 Consider the region enclosed by the curve

x2 + y 2 = (x2 + y 2 x)2 .
Show that in polar coordinates, this curve is given by
r = 1 + cos .
Sketch the curve, and compute the area it encloses.
7.11 Consider the closed curve given in polar terms by r = sin3 . Sketch this curve, and compute
the area enclosed.
7.12 Consider the closed curve given in polar terms by r = 1 + sin(2). Sketch this curve, and
compute the area enclosed.
7.13 Let D be the region in R2 that is bounded by the lines
yx=0
yx=3
Compute
Z
D
x2
x + 2y = 2
and x + 2y = 4 .
xy
dA .
+ 4xy + 4y 2
7.14: Let D R2 be the region bounded by xy = 1, xy = 2, x2 y = 3 and x2 y = 4. Compute

R
xydA.
D
7.15: Let D be the set in R2 that is given by
1
Let f (x, y) =
1
x2 y 2
. Compute
R
D
y
2
x2
and
x
2.
y2
f (x, y)dA.
7.16: Let D be the set in R2 that is given by

0 x2 y 2 4
and
1 xy 2 .
7.5. EXERCISES
323
Let f (x, y) = x2 + y 2 . Compute
R
D
f (x, y)dA.
7.17 Let V be the region in R3 that is bounded above by the sphere x2 + y 2 + z 2 = 4, and below by
p
R
the cone 4z = 4 x2 + y 2 . Let f (x, y, z) = 1/(x2 + y 2 + z 2 )2 . Compute V f (x, y, z)dV .
7.18: Let V be the region in R3 that is the intersection of the three cylinders of unit radius along
the three coordinate axes. That is, D is the set of points (x, y, z) satisfying
y2 + z2 1
x2 + z 2 1
and x2 + y 2 1 .
Compute the volume of V.

7.19: Let V be the region in R3 that is above the sphere x2 + y 2 + z 2 = 6 and below the paraboloid
z = 4 x2 y 2 . Compute the volume of this region.
7.20: Let V be the region in R3 that is bounded by the surfaces
p
x2 + y 2
= z2
x2 + y 2
8 z2
Compute the volume of V and the total surface area of its boundary. (There are two pieces to the
boundary.
(a) Draw a plot of the intersection of the bounding surfaces with the x, z plane.
(b) Compute the volume of V.
(c) Compute the total surface area of the boundary of V.
7.21: Let V be the region in R3 consisting of points (x, y, z) satisfying
0 z y 2 sin x
for 0 x , and 0 y 1. Find the average height in V.
7.22: Let V be the region in R3 bounded by y + z = 2, 2x = y x = 0 and z = 0. Let f (x, y, z) = xez .
R
Compute V f (x)dV .
7.23: Let V be the region in the positive octant of R3 bounded by the elliptic cylinder 9x2 + 4y 2 = 36
R
and the sphere x2 + y 2 + z 2 = 16. Let f (x, y, z) = z. Compute V f (x)dV .
7.24: Let V be the region in R3 contained in the cylinder x2 + y 2 = 9, and lying below the plane
y = z, and above the plane z = 0. Compute the volume of V.
7.25: Let V be the region in R3 lying between the paraboloids z = x2 + y 2 and z = 8 x2 y 2 .
Compute the volume of V.
7.26: Let S be the triangle in R3 with vertices (0, 4, 1), (1, 0, 2) and (0, 0, 3).
(a) Find a parameterization of the surface S. (Hint: Find an equation for the plane containing .
R
(b) Let f (x, y, z) = xyz. Compute S f dS.
7.27: Let S be the surface in R3 given by z = xy with x2 + y 2 1.
(a) Find a parameterization of the surface S.
R
(b) Let f (x, y, z) = z 2 . Compute S f dS.
324
7.28: Let S be the torus in R3 obtained by rotating the circle of radius a on the x, z plane centered
at (b, 0, 0), where b > a, about the z-axis.
(a) Find a parameterization of the surface S.
(b) Compute the surface area of the torus.
7.29: Let x(r, ) := (r cos , r sin , g(r)) for 0 2 and, for given positive numbers a and b,
a r b. Show that the area of the corresponding parameterized surface S is
Z
2
a
1 + (g 0 (r))2 rdr .

Multivariable Calculus, Linear Algebra and Differential Equations

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Multivariable Calculus, Linear Algebra and Differential Equations

Hochgeladen von

Copyright:

Verfügbare Formate

MULTIVARIABLE CALCULUS, LINEAR

ALGEBRA AND DIFFERENTIAL EQUATIONS

Algebra and geometry in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Vector variables and Cartesian coordinates . . . . . . . . . . . . . . . . . . . .

The vector space R

Geometry and the dot product . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Parallel and orthogonal components . . . . . . . . . . . . . . . . . . . . . . . . 19

Householder reflections and orthonormal bases . . . . . . . . . . . . . . . . . . 24

Lines and planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

The cross product in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Equations for planes in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Equations for lines in R3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2 DESCRIPTION AND PREDICTION OF MOTION

Functions from R to R and the description of motion . . . . . . . . . . . . . . . . . . 61

Continuity of functions from R to Rn

Differentiability of functions from R to Rn . . . . . . . . . . . . . . . . . . . . . 63

Velocity and acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Torsion and the FrenetSeret formulae for a curve in R3 . . . . . . . . . . . . . 72

Curvature and torsion are independent of parameterization. . . . . . . . . . . . 79

Speed and arc length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

The prediction of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Newtons Second Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Ballistic motion with friction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Motion in a constant magnetic field and the Rotation Equation . . . . . . . . . 87

The specification of curves through differential equations . . . . . . . . . . . . . 97

Continuity in several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Functions of several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Continuity in several variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Continuity and limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

The squeeze principle is several variables . . . . . . . . . . . . . . . . . . . . . . 110

Continuity versus separate continuity

Continuity, compactness and maximizers . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Open and closed sets in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Minimizers and maximizers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Compactness and existence of maximizers . . . . . . . . . . . . . . . . . . . . . 120

Vertical slices and directional derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Directional derivatives and partial derivatives . . . . . . . . . . . . . . . . . . . 129

The geometric meaning of the gradient . . . . . . . . . . . . . . . . . . . . . . . 135

Critical points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

The gradient and tangent planes . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Linear functions from Rn to Rm

The matrix representation of linear functions . . . . . . . . . . . . . . . . . . . 146

Solving the equation Ax = b for m = n . . . . . . . . . . . . . . . . . . . . . . 151

The distance between two matrices . . . . . . . . . . . . . . . . . . . . . . . . . 155

Differentiability of functions from Rn to Rm . . . . . . . . . . . . . . . . . . . . . . . . 159

Differentiability and best linear approximation in several variables . . . . . . . 159

The general chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Linear approximation and Newtons iterative scheme . . . . . . . . . . . . . . . 166

The Geometry of Newtons Method . . . . . . . . . . . . . . . . . . . . . . . . 168

Starting and stopping the iteration . . . . . . . . . . . . . . . . . . . . . . . . . 170

The structure of linear transformations from Rn to Rm . . . . . . . . . . . . . . . . . . 174

The Fundamental Theorem of Linear Algebra . . . . . . . . . . . . . . . . . . . 174

Linear independence and Gram-Schmidt orthonormalization . . . . . . . . . . . 175

Computing rank and solving linear equations . . . . . . . . . . . . . . . . . . . 179

5 THE IMPLICIT FUNCTION THEOREM AND ITS CONSEQUENCES

Horizontal slices and contour curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Implicit and explicit descriptions of planar curves . . . . . . . . . . . . . . . . . 192

The tangent line to a contour curve . . . . . . . . . . . . . . . . . . . . . . . . 195

When is the contour curve actually a curve? . . . . . . . . . . . . . . . . . . . . 196

Constrained Optimization in Two variables . . . . . . . . . . . . . . . . . . . . . . . . 200

Lagranges criterion for optmizers on the boundary . . . . . . . . . . . . . . . . 202