Sie sind auf Seite 1von 12

Numerical differentiation

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 1 / 12


Numerical differentiation

In general, symbolic differentiation is very efficient.


Sometimes : no analytical form available
Ex : Implicit functions, measures,. . .
Simple formulas but . . .
. . . huge roundoff errors

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 2 / 12


Naive method (order 1)

Just apply the definition


Write a table for h going to 0

f (x + h) − f (x)
Fh :=
h

Proposition
Let f be a function twice continuously differentiable, for which we want to
compute the first derivative in x. We define the error as
E (h) := f (x+h)−f
h
(x)
− f 0 (x). For all h, there exists ξ ∈ [x, x + h] and C > 0 such
that
C
|E (h)| ≤ h.
2

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 3 / 12


Practical example

Let us compute the derivative of f (x) = x 4 in x = 1


Theoretical result : f 0 (x) = 4x 3 and f 0 (1) = 4.
In practice :
Best approximation : 8 correct digits

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 4 / 12


Central difference formula

Symmetric around x
Tabulate, for h going down to 0

f (x + h) − f (x − h)
Fh :=
2h

Proposition
Let f be a function three times continuously differentiable. The sequence
f (x+h)−f (x−h)
2h converges to f 0 (x) with an order of convergence of 2.

Practical example
f (x) = x 4
Up to 11 correct digits

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 5 / 12


One-sided formulas

Goal : Exceptionnally we sometimes only have values on one side of the interval.

Methodology
To find a one-sided formula and its order of convergence :
Write the Lagrange formula that yields the interpolation polynomial
Compute the derivative of the formula
Use the Taylor expansion to find the order of convergence

Examples
One-sided formulas of order 2
Two-sided formula for the second derivative
An unsolved issue : we do not get rid of roundoff errors !

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 6 / 12


Optimal choice of the step h
If h increases → the theoretical error increases and the roundoff errors
decrease
If h decreases → the theoretical error decreases and the roundoff errors
increase
How to find the right balance ?

Proposition
For a centered formula, the total error (theoretical+roundoff) is approximately
given by
2C M D
|E (h)| ≈ + h2 .
h 6
Roundoff error Theoretical error

Optimal choice of the step h


For the centered formula, r
3 12C M
h= .
Quentin Louveaux (ULg - Institut Montefiore)
2D
Numerical analysis 2018 7 / 12
Richardson extrapolation

How to obtain the best possible precision and get rid of roundoff errors. . .

Centered differences
f (x+h)−f (x−h)
Step h 2h |E (h)|
10−1 4.0400000000000018 0.0400000000000018
10−2 4.0004000000000035 0.0004000000000035
10−3 4.0000039999997234 0.0000039999997234

We do see that the sequence converges to 4 until something wrong happens.

Idea
Knowing that the sequence converges quadratically, we can extrapolate from the
beginnning of the sequence what is the end of the sequence.
This way, we get rid of the roundoff errors

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 8 / 12


Richardson extrapolation

We assume that we have

gh = g + c1 h + c2 h2 + c3 h3 + · · ·

Thus gh converges linearly to g i.e.

lim gh = g .
h→0

Theorem
The sequence 2g h − gh converges quadratically to g .
2

Alternatively, the sequence 10g h − gh converges quadratically to g .


10
We can also go on and obtain a sequence that converges cubically.

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 9 / 12


Application to the numerical computation of the derivative

The centered difference formula gh converges already quadratically to g :

gh = g + c2 h2 + c4 h4 + c6 h6 + · · ·

Proposition
g h − 14 gh
The sequence 2
3 converges to g with an order of convergence of h4 .
4

Examples
Computations of f 0 (1) for f (x) = x 4 : too easy for Richardson extrapolation !
Computation of h0 (0) for h(x) = sin x.

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 10 / 12


Array representation of the Richardson extrapolation
General formula
We denote by Gi,j the approximations of g where we got rid of the term of order j
(order j + 1 approximation) and where the approximation uses the values
h , . . . g h . The values Gi,j from the Richardson extrapolation are given by
g i−j i
2 2

Gi,j−1 − 21j Gi−1,j−1


Gi,j = .
1 − 21j

Table
h G0,0
&
h
2 G1,0 → G1,1
& &
h
4 G2,0 → G2,1 → G2,2
& & &
h
8 G3,0 → G3,1 → G3,2 → G3,3
& & & &
h
16 G4,0 → G4,1 → G4,2 → G4,3 → G4,4
Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 11 / 12
Table for the computation of the derivative

Example : computation of the derivative of e x in 0.


h Gi,0 Gi,1 Gi,2 Gi,3
2−1 1.0421906109874948
2−2 1.0104492672326730 0.9998688193143991
2−3 1.0026062019289235 0.9999918468276737 1.0000000486618921
2−4 1.0006511688350699 0.9999994911371187 1.0000000007577483 0.9999999999973651
2−5 1.0001627683641381 0.9999999682071609 1.0000000000118303 0.9999999999999903

Quentin Louveaux (ULg - Institut Montefiore) Numerical analysis 2018 12 / 12