Numerical differentiation

Dr.U.Oral nal
 Polynomial Approach
According to Newtons fundamental formula:

f x Pn x Rn x
Hence we can write:

df x dPn x dRn x

dx dx dx
 Polynomial Approach
Forward differentiation

nth order forward finite difference polynomial can be given as:

1 1 n 1
Pn f 0 f 0 2 f 0 n f 0
2! n!
x x0
where
h

and the error term is:

1 2 n 1 2 n
Rn x0 h h n 1 f ( n 1) ( ) n 1 f 0
n 1! n 1!
 Polynomial Approach
Forward differentiation
For example, three-point formula may be derived as follows:
dP2 ( x) dP2 ( ) d 1 2 1 2
f f0
d dx
0
dx x x0 x x0 h 2 0
1
f f
1
f 2 f f
h
1 0 2 1 0
2

1
f 2 4 f1 3 f 0
2h
 Polynomial Approach
Backward differentiation
By using backward finite differences and considering nth order polynomial,
1 1 n 1
Pn f 0 f 0 2 f0 n f0
2! n!
For instance, the first order three-point backward differentiation formula can
be obtained as follows:

dP2 dP2 d 1 2 1 2
f f0
d dx h
0
dx x x0 0 2! 0
1
f f
1
f 2 f f
h
0 1 0 1 2
2

1
3 f 0 4 f 1 f 2
2h
 Polynomial Approach
Central differentiation
Using Stirling`s formula:

2 ( ) 3
0 0
0 + = 0 + 0 + 2 + 1 +1 +
2! 3!
where 1/2(E1/2+E-1/2)
 Polynomial Approach
Central differentiation

In order to derive the first central differentiation formula, the following

operations can be made:

dP2
dx

1
h

f 0 2 f 0
0
x x0

dP2 1 1 h h
f x0 f x0 ( f 0 )
dx x x0 h 2 2 2 0
1 1
f1 f 1 f1 2 f 0 f 1
dP2
dx x x0 h 2 0
dP2 f1 f 1

dx x x0 2h
 Error term
The error for the differentiation formulas may be determined with the aid of the
error term Rn(x):

n f n 1
Rn x x xi ; x0 xn
i 0 n 1!
dRn d x f n 1 1 f n 2 2
x
dx dx n 1! n 2!
where

n
x x xi and 1 , 2 x0 , x1 , xn
i 0
 Error term
dRn
If we evaluate at x=xj, since the second term is zero, we can obtain:
dx

dRn x j d x j f n 1 n f n 1
x j xi
dx dx n 1! i 0 n 1!
i j
d x j
dx is of the order of hn. From this point of view, since the value of
f(n+1)() is unknown for most cases, we can express the error on the numerical
differentiation in terms of the order of the power of the interval.
However, it should be born in mind that the rounding error increases with
decreasing values of h.
dRn x
dx

O hn

d 2 Rn x d m Rn x
dx 2

n 1
O h ,,
dx m

O h n m 1
 Example
Find the value of y at x=0.0 by using the value pairs given below.
Determine the truncation and rounding errors.
xi yi y 2 y 3 y 4 y
0.00 0.00000
0.10017
0.05 0.10017 0.00100
0.10117 0.00101
0.10 0.20134 0.00201 0.00003
0.10318 0.00104
0.15 0.30452 0.00305 0.00003
0.10623 0.00107
0.20 0.41075 0.00412
0.11035
0.25 0.52110
 Example
By using a fourth degree polynomial ;
1 1 2 1 2 3
P4 f 0 f 0 2 f 0 3 f 0 4 f 0
2! 3! 4!
dP4 1 dP4 1 2 1 2 3 2 6 2 3 4 3 18 2 22 6 4
f 0 f0 f0 f0
dx h d h 2 6 24 0
dP4 1 1 1 1
f 0 2 f 0 3 f 0 4 f 0 1.99998
dx x x0 h 2 3 4

may be found.
On the other hand, the same result may be obtained by using finite difference
expressions, without generating a difference table:

1
dP4
f f
1
f 2 f f
1
f 3 f 3 f f
1
f 4 f 6 f 4 f f
h
1 0 2 1 0 3 2 1 0 4 3 2 1 0
dx x x0 2 3 4
dP4

1
25 f 0 48 f1 36 f 2 16 f 3 3 f 4 1.99998
dx x x0 12h

The derivative expression found above is the fourth degree forward differentiation
formula.
 Example
Now let us examine the truncation and rounding errors:
dR4 1 d 1 2 3 4 R x x x x x x x
f n 1

5
f 0

dx x x h d n 1
(n 1)!h
n 0 1 n
0
0
5! 0

24 1 5 1 5 1 2 n
f0 f0 Rn h x0 n 1 f 0
5! h 5h n 1!

Although h=0.05 and 5f0 is 0.00000, there exists, at least, a rounding error of
0.000005. Hence the truncation error may be found as:
dR4
2 105
dx x x0

This result is also validated by the error expression previously given:

dR4
dx

O h 4 0.625 10 5
x x0

O 10 5
 Example
On the other hand, the rounding error of each yi is yi=0.000005=. Because of
the 128 operations performed in the fourth forward differentiation formula the
rounding error is high.
In order to find the rounding error made in the calculation of y, let us first
consider the relative error:
128 25 f 0 48 f1 36 f 2 16 f 3 3 f 4
y num denom 12 h

The error of 12 and h may be taken as zero since they are accurate values. The
square bracket represents the numerator of y.
128
Therefore, since the relative error is y , the absolute error is:

y
128

12h As is seen, the rounding error is greater than the truncation
error and it is inversely proportional with h. Accordingly, it
128 0.000005
y 10 3 is more accurate to give the value of the derivative as:
12 0.05
y x 0 2.00
 Order of Accuracy for the Central Differences

3-point ------ First derivative ----- Second-order accuracy

3-point ------ Second derivative ----- Second-order accuracy
5-point ------ First derivative ----- Fourth-order accuracy
5-point ------ Second derivative ----- Fourth-order accuracy
5-point ------ Third derivative ----- Second-order accuracy
5-point ------ Fourth derivative ----- Second-order accuracy

Example:
Using the polynomial approach, calculate;
o the third derivative with first order accuracy, ileriye doru farklar ile,
o The second derivative with second-order-accuracy central differences.
14
 Differentiation with the Finite Difference Operators
High-order formulas can be obtained by using finite difference operators:

n is even

n is odd

Example:
Use the formulas given to obtain;
o the third derivative with first order accuracy, ileriye doru farklar ile, 15
o The second derivative with second-order-accuracy central differences.
 Differentiation with Taylor series
-Forward Differentiation-
f (x+x) can be expanded to Taylor series about x as follows:

16
 Differentiation with Taylor series
-Backward Differentiation-
Expanding f(x-x) about x:

17
 Differentiation with Taylor series
-Backward Differentiation-
Taking the difference of f(x-x) and f(x+x):

18
 Differentiation With Taylor Series
Higher Order Derivatives-
-

Once again, let us use Taylor series expansion:

(a)

(b)

Multiply (a) by 2 and subtract it from (b):

This is the forward difference approximation of the second derivative. 19

 Differentiation With Taylor Series
- Higher Order Derivatives-
In a similar manner, the backward difference approximation of the second
derivative can be obtained:

In order to obtain the central difference formula, simply add the forward and
backward approximation formulas:

20
 Differentiation with Taylor series
-Higher Order Accuracy-

Solving for df/dx:

Substitute a forward difference expression for d2f/dx2, for instance:

One can obtain:

21
 Example
- 4-point backward formula (Dawson) -

For a 1-dimensional grid system let us expand the u velocities to Taylor series
 Example
- 4-point backward formula (Dawson) -

Multiply the first equation by 15, the second one by (-6) and collect three
equations:

which gives:
 Extrapolation
Suppose that we have tabulated values.

With central differencing:

This time using a larger h value:

Assuming that C1=C2

 Extrapolation
Generally, we can extrapolate to a better estimate of the exact value:

We can also repeat the scheme using h values of 0.2 and 0.4.

The exact value is 0.128171

 Extrapolation
In the same manner, the second order derivatives can also be calculated.
By using the tabulated values previously given :

The exact value is 0.13379

The rounding error is critical!

 Extrapolation
If the function is known then the same method can be applied by using
smaller h values than that initially selected. This technique is known as
Richardsons extrapolation.
Let us build a Richardson table for the function given below to evaluate

The exact value is 0.239132

Method of Undetermined Coefficients

We should first specify the number of points used in the formula and
indicate whether the formula is a forward, backward or central-
difference formula.

Let us obtain the three-point central-difference formula:

Expanding f(x0-h) and f(x0+h) in Taylor series:

28
Method of Undetermined Coefficients

Collecting like terms and simplifying:

We should have:

Hence:

29
 Rounding and truncation
Calculating the first derivative with the three-point forward
differentiation:

For each value we have a rounding error:

Total error:
 Rounding and truncation
For the rounding errors we have:

Assuming that

Taking the derivative:

Partial Derivatives of Multivariable Functions
Let us consider this model equation

Time, x and ve y directions

can be discretised,
respectively as follows:

32
 Partial Derivatives of Multivariable Functions

Hence, one can write:

33
 Mixed Partial Derivatives
First let us perform a forward
differentiation with respect to y:

Then two differentiations (for both points) with respect to x

are required. With forward differentiation:

34
 Mixed Partial Derivatives
Let us consider the same case. Now we use
the second-order central differences.

And partial differentiation with respect to x:

35
What Happens When the Node Points Are Not
Equidistant?
How can we calculate the derivatives in the following case?

Let us recall divided differences:

f x1 f x0
f x1 , x0 f x0 , x1
x1 x0

f x2 , x1 f x1 , x0
f x2 , x1 , x0 f x1 , x0 , x2 f x0 , x2 , x1
x2 x0 36
 What Happens When the Node Points Are Not
Equidistant?

P2 x f x0 x x0 f x1 , x0 x x0 x x1 f x2 , x1 , x0

2
= 1 , 0 + 2 1 0 2 , 1 , 0
=0
=0

= 1 + 0 1 2[ ]
2 1 1 0
2 , 1 1 , 0 2 1 1 + + 0
2 = 2 , 1 , 0 = = =
2 0 (1 + ) (1 + ) 2

10 21 1+ +0 0 2+ + 1+ 2 12
=
(1+) 2 (1+)

+ 2 + + 1 2 +1 +2
=
(1 + )
37