9/15/2009
Introduction
Numerical Differentiation
eating
Approximating the derivative
The definition of the derivative, the limit as
h→0 of
f  x  h   f x 
f ' x  
h
practically suggests the program to pick a
small value of h.
Example. Compute the derivative of f(x)=ex
at x=1.
Approximating the derivative
• When h is large, the difference quotient is
the slope of the secant line joining two
points that are not close enough to each
other.
• When h is too small, the difference
quotient can exhibit the problem of loss of
significance due to the subtraction of
quantities that are nearly equal.
• Some numerical algorithms require
evaluation of the derivative of a function.
• When the function is difficult to
differentiate analytically or when the
function is given empirically as in table of
values, approximate calculation of the
derivative is necessary.
Limit of a difference quotient
Derivative of f(x)=ex at x=1 for different values of h.
(1)
Approximating the derivative
• To determine the best approximation
before the terms start to move away from
the limit is to compute Dk until
DN 1  DN  DN  DN 1
• In the example,
0.0007  D6  D5  D5  D4  0.00012
1
 9/15/2009

Sources of error Central-difference formulas

• The truncation error t comes from higher • Assume that f C3[a,b] and that x-h, x, x+h
terms in the Taylor series expansion: [a,b]. Then
f x  h   f x  1 f x  h   f x  h 
f '  hf "  f ' x    t ( 2)
h 2 2h
• Round-off error r must be expected since where, there exists a number c=c(x) [a,b]
real numbers don’t have exact such that
representation in binary. h 2 f 3  c 
t  
6
 
 O h2

Centered-formula of order O(h4) Example

Assume that f C5[a,b] and that x-2h, x-h, x Let f(x)=cos(x). Calculate approximations for
x+h, x+2h [a,b]. Then f’(0.8) using Eqs. (2) and (3) with step
sizes h=0.1, 0.01, 0.001, 0.0001 and
 f  x  2h   8 f  x  h   8 f  x  h   f  x  2 h 
f ' x   t compare with true value f’(0.8) = -sin(0.8).
12h
(3)
where Carry out nine decimal places in
h 4 f 5  c  calculations.
t  
30
 O h4  

Differentiating an
Example
Interpolating Polynomial
Eq. (2) • The Lagrange quadratic polynomial P2(x)
f x  h    x  h 
f ' x   that passes through x=0.7,0.8, and 0.9.
2h
Eq. (3)
P2  x   1.046875165  0.159260044 x  0.348063157 x 2
 f  x  2h   8 f  x  h   8 f  x  h   f  x  2h 
f ' x  
12h
• Differentiating

P2 ' 0.8  0.716161095

2
 9/15/2009

Differentiating an
Differentiation using Limits
Interpolating Polynomial
• The Lagrange quadric polynomial P4(x) To approximate f ’(x) numerically, generate a
that passes through x=0.6,0.7,0.8, 0.9 and sequence
1.0.
f '  x   Dk 
  
f x  10  k h  f x  10 k h 
P4  x   0.998452927  0.00963839 x  0.523291341x 2
 
2 10k h
 0.026521229x 3  0.028981100x 4
for k=0,..,n until |Dn+1 – Dn| ≥ |Dn-1 – Dn| or
• Differentiating |Dn – Dn-1|≤ tolerance.
P4 ' 0.8  0.717353703

Richardson’s Extrapolation Richardson’s Extrapolation

Let Manipulating these two equations, we obtain
f k  f xk   f  x0  kh  4 D0 h   D0 2h   f 2  8 f1  8 f 1  f 2
f ' x0   
and D0(h) and D0(2h) denote 3 12h
approximations to f ’(x0) that are obtained which is the central difference formula
using Eq. (2) with step sizes h and 2h. Eq.(3).

f ' x0   D0 h   Ch 2 This method of obtaining a formula for f ’(x0)

f ' x0   D0 2h   4Ch 2
4 of higher order from a formula of lower
order is called extrapolation.

Richardson’s Extrapolation Example

Suppose that two approximations of order Let
O(h2k) for f ’(x0) are Dk-1(h) and Dk-1(2h) and f(x)=cos(x).
that they satisfy Use Eq. (4) with h=0.01 to show how the
f '  x0   Dk 1 h   c1h 2k  c2 h 2k 2   linear combination (4D0(h) -D0(2h))/3 can
f '  x0   Dk 1 2h   4 k c1h 2 k  4 k 1 c2 h 2 k  2   be used to obtain the approximation to
Then an improved approximation is f’(0.8).
4 k Dk 1 h  Dk 1 2h Carry nine decimal places in all
 
f '  x0   Dk h   O h 2 k 2 
4k  1

 O h 2 k 2  calculations.

3
 9/15/2009

Example Higher Derivatives

Using Eq. (4), we obtain • Taylor series can be used to obtain
D h  
f 0.81  f 0.79 
 0.717344150
central-difference formulas for higher
0
0.02 derivatives.
f 0.82   f 0.78
D0 2 h    0.717308275
0.04
The linear combination becomes
• The popular choices are those of order
4 D0 h   D0 2h  O(h2) and O(h4).
f ' 0.8    0.717356108
3

Higher Derivatives Higher Derivatives

To derive f ”(x) of order O(h2), we write Solving for f ”(x) yields
h 2 f "  x  h 3 f 3   x  f x  h   2 f x   f x  h 
f x  h   f x   h  f ' x     f " x  
2 6 h2
h 2 f "  x  h 3 f  3  x  2h 2 f 4   x  2h 4 f 6   x  2h 2 k 2 f  2 k   x 
f x  h  f x   h  f ' x        
2 6 4! 6! 2k !
adding If truncated at the fourth derivative
2 h 2 f "  x  2h 4 f  4   x  f1  2 f 0  f 1 h 2 f 4  c 
f x  h  f x  h  2 f x     f " x   
2 24 h2 12

Central difference formulas Central difference formulas

of Order O(h2) of Order O(h4)
f1  f 1
f '  x0   f '  x0  
 f 2  8 f1  8 f 1  f  2
2h 12h
f  2 f 0  f 1  f 2  16 f1  30 f 0  16 f 1  f  2
f "  x0   1 f " x0  
h2 12h 2
 f 3  8 f 2  13 f1  13 f 1  8 f  2  f 3
f  f 2f  f f 3   x0  
f 3  x0   2 1 3 1 2 8h3
2h  f3  12 f 2  39 f1  56 f 0  39 f 1  12 f 2  f 3
 f 4  x0  
f 4 f1  6 f 0  4 f 1  f 2 6h 4
f  4   x0   2
h4

4
 9/15/2009

Example Example
Let f(x)=cos(x). Use the formula • The calculation for h=0.01 is
f1  2 f 0  f 1 f 0.81  2 f 0.80  f 0.79 
f "x  f " 0.8   0.696690000
h2 0.0001

with h=0.1, 0.01, and 0.001 to find with an error of -0.000016709

approximations to f ”(0.8). • The other calculations are summarized:
Compare with the true value of f ”(0.8) =
-cos(0.8).

Exercise 1 Exercise 2
Let f(x)=sin(x), where x is measured in Use numerical differentiation formula
radians.
(a) Calculate approximations to f ’(0.8)  f 2  16 f1  30 f 0  16 f 1  f  2
f "  x0  
using Eq.(3) with h=0.1, 0.01, and 12h 2
compare with f ’( 0.8)=cos(0.8).
and h=0.1 to approximate f ”(1) for the
(b) Use extrapolation formula to compute functions
the approximation to f ’( 0.8)
(a) f(x) = x2 (b) f(x) = x4