Optimisation

For y = f(x) the differential is defined as

Definition

Total Differential

If we move from (x0, y0) to a point (x0 + dx, y0 + dy) nearby, the
resulting change
=

In the linearization of f is called the total differential of f

Example: The volume =

of a cylinder is to be calculated
from the measured values of r (the radius) and h (the height).
Suppose that r is measured with an error of no more than 2% and h
with an error of no more than 0.5%. Estimate the resulting possible
percentage error in the calculation of V.
First note that

100 0.5

100 2,
Then

=
=
= 2

+
2

+
2

Therefore error no more than 4.5%

=2
+

+
2

+
2 0.02 + 0.005 = 0.045

Extreme Values and Saddle Points

In Eng Maths 1 when we investigated extreme values for functions
of one variable we looked for points where the graph had a
horizontal tangent line.
For functions of two variables we look for points where the surface
defined by z = f(x,y) has a horizontal tangent plane. This leads to
the following definition:Definitions

Local Maximum, Local Minimum

Let f(x, y) be defined on a region R containing the point (a, b). Then
1. f(a, b) is a local maximum value of f if f(a, b) f(x, y) for all
domain points (x, y) in an open disk centered at (a, b).
2. f(a, b) is a local minimum value of f if f(a, b) f(x, y) for all
domain points (x, y) in an open disk centred at (a, b)

Local maxima correspond to mountain peaks on the surface

z = f(x, y) and local minima correspond to valley bottoms,

Theorem: First Derivative Test for Local Extreme Values

If f(x, y) has a local maximum or minimum value at an interior
point (a, b) of its domain and if the first partial derivatives exist
there then fx(a, b) = 0 and fy(a,b) = 0

Definition

Critical Point

An interior point of the domain of a function f(x, y) where both fx and

fy are zero or where one or both of fx and fy do not exist is a critical
point of f.
Therefor local maxima and minima are critical points but critical
points can also include saddle points
Definition

Saddle Point

A differentiable function f(x, y) has a saddle point at a critical point

(a, b) if in every open disk centred at (a, b) there are domain points
(x, y) where f(x, y) > f(a, b) and domain points (x, y) where f(x, y) <
f(a, b). The corresponding point (a, b, f(a, b)) on the surface
x = f(x, y) is called a saddle point of the surface.

An example of a saddle point is the origin in the following surface:

Therefore, finding critical points of a function it is not sufficient

to identify the type of critical point (local maximum, local
minimum or saddle point). To do this we need to make use of
second partial derivatives.

Theorem: Second Derivative Test for Local Extreme Values

Suppose that f(x, y) and its first and second partial derivatives are
continuous throughout a disk centred at (a, b) and that
fx(a, b) = fy(a, b) = 0. Then
1.

f has a local maximum at (a, b) if fxx < 0 and

fxxfyy fxy2 > 0 at (a, b).

2.

f has a local minimum at (a, b) if fxx > 0 and

fxxfyy fxy2 > 0 at (a, b).

3.

f has a saddle point at (a, b) if fxxfyy fxy2 < 0 at (a, b).

4. the test is inconclusive at (a, b) if fxxfyy fxy2 = 0 at (a, b). In

this case, we must find some other way to determine the
behaviour of f at (a, b)

The quantity

is called the discriminant or Hessian of the function f.

Example: Find the local extreme values of

, =

2 2 +4
and determine the nature of each.
f(x, y) is defined and differentiable for all points in its domain.
Hence at extreme values fx and fy are simultaneously zero. This gives
the two equations
=

2 2 = 0

2 2=0

The solution of these equations is x = y = -2. Hence (-2, -2) is the

only point where f may take an extreme value. Now take the second
derivatives
= 2 < 0

= 2

=1

At the point (-2, -2)

So fxx < 0 and

= 2 2 1 = 3 > 0

> 0.

Therefore f has a local maximum at (-2, -2). The value at

this point is f(-2, -2) = 8.
The value of f at this point is f(-2, -2) = 8.

Summary of Max-Min Tests

The extreme values of f(x, y) can occur only at
1. boundary points of the domain of f
2. critical points (interior points where fx = fy = 0 or points
where fx or fy fail to exist)
3. If the first- and second-order partial derivatives of f are
continuous throughout a disk centred at a point (a, b) and
fx(a, b) = 0, the nature of f(a, b) can be tested with the
Second Derivative Test:
i)
ii)
iii)
iv)

fxx < 0 and fxxfyy fxy2 > 0 at (a, b) local maximum

fxx > 0 and fxxfyy fxy2 > 0 at (a, b) local minimum
fxxfyy fxy2 < 0 at (a, b) saddle point
fxxfyy fxy2 = 0 at (a, b) test inconclusive

Lagrange Multipliers
We now consider the problem to find extrema of a function
f(x, y, z) whose domain is contrained by another function
g(x, y, z) = 0 to lie within some subset.

Suppose that f(x, y, z) and g(x, y, z) are differentiable and

0
when g(x, y, z) = 0. To find the local maximum and minimum
values of f subject to the constraint g(x, y, z) = 0, we need to find
the values of x, y, z and that simultaneously satisfy the equations
=

and

, ,

=0

This is the Method of Lagrange Multipliers. For functions of

two variables the condition is similar but without the variable z.

Example: Find the greatest and smallest values that the function
f(x, y) = xy takes on the ellipse

=1

We need to find the extreme values of f(x, y) = xy subject to the

constraint
,

1=0

First, find the values of x, y and for which

=

and
=

, ,

Hence
+

Comparing components gives

=

=0

Hence y = 0 or = 2 and there are two cases to consider.

1. If y = 0, then x = y = 0. But (0, 0) does not lie on the ellipse, hence
y 0.
2. If y 0, then = 2 and x = 2y. Substituting in g(x, y) = 0 gives
2
8
4

+
+4

=1

=8

= 1

Therefore f(x, y) has its extreme values on the ellipse at the four
points (2, 1), (2, -1).
The extreme values are xy = 2 and xy = -2.
The level curves of f(x, y) = xy are the hyperbolae xy = c. The
extreme values are the points on the ellipse when
(red) is a scalar
multiple of
(blue):

Example: Find the maximum and minimum values of the function

, = 3 + 4 on the circle
+
= 1.
,

= 3 + 4

The Lagrange multiplier condition states that

3 +4 =2

+2

3
2
=
= ( 0)
2

Therefore x and y have the same sign.

The condition g(x, y) = 0 gives
+

1=0

, hence

and this gives

3
2

1=0

9
4
+ = 1 9 + 16 = 4
4

5
=
2

3
3
2
4
=
= = =
2
5
5
Therefore the function f(x, y) = 3x + 4y are the lines 3x + 4y = c.
The farther the lines lie from the origin, the larger the absolute
value of f.