Theoretical tools

Taylor’s theorem
Theorem 1 Let y = y(t) be a function of one variable and f = f (t, y(t)) be a function
of two variables with continuous first partial derivatives on the rectangular domain Ω =
I × [y0 − s, y0 + r], r, s > 0, t ∈ I. Then the following holds:

(i) For any t1 , t2 ∈ I there exists a point τ ∈ [t1 , t2 ] such that

(t2 − t1 )2 00
y(t2 ) = y(t1 ) + (t2 − t1 )y 0 (t1 ) + y (τ ).
2

(ii) For any points (t, y1 ), (t, y2 ) ∈ Ω there exists a point (t, η) ∈ Ω such that

f (t, y2 ) = f (t, y1 ) + (y2 − y1 )fy (t, η).

Examples of Taylor expansions

h2 00 h3 000 h4 (iv)
y(t + h) = y(t) + hy 0 (t) + 2
y (t) + 3!
y (t) + 4!
y (t) + ···
h2 00 h3 000 h4 (iv)
y(t − h) = y(t) − hy 0 (t) + 2
y (t) − 3!
y (t) + 4!
y (t) + ···
h2 00 h3 000 h4 (iv)
y(t) = y(t + h − h) = y(t + h) − hy 0 (t + h) + 2
y (t + h) − 3!
y (t + h) + 4!
y (t + h) + · · ·
For a function in more than one variable, one can apply Taylor expansion with respect to
only one of the variables or with respect to all (which latter case we do not consider here.
For f (t, y) we use case (ii) from Taylor’s theorem.

Truncation and discretization errors

At present, in the literature regarding numerical methods for ODEs there is no consensus
regarding the definitions of the two important notions, related to estimating the errors
when computing a numerical solution of a given continuous ordinary differential equation,
namely, the truncation and the discretization errors. Various books interchange their
names, which creates a source of misunderstanding and confusion.

We present below a simplified way to define the errors, suited for some simple methods,
such as one-step methods. The framework we follow here is usually discussed in more
detail in the context of partial differential equations and initial boundary value problems,
see for instance, [1], [2].

1
Consider a differential operator L(y), for example, L(y) ≡ y 0 . We pose the task to compute
y as the solution of the differential equation

L(y) = f (t, y(t). (1)

We assume also that f is Lipschitz continuous and the equation has a unique solution.
Let L(y) be a difference operator that approximates L in some way, e.g.,
1
L(y) ≡ (y(t + h) − y(t). (2)
h

To find the solution of (1) in analytical form is in general possible in very few cases and
for simple equations. Therefore, we seek an approximate solution, computed in a finite
number of points, tk , k = 0, 1, · · · , n using some numerical method.
The numerical method is usually defined by replacing the continuous differential operator
L by some discrete operator L and then we seek the approximate (numerical) solution z,
that satisfied the difference equation

L(z) = F (t, z), (3)

where F (t, y) is related to a particular numerical method.

Definition 1 We define the so-called truncation error at point t as the error by which the
exact solution fails to satisfy the difference equation (3), namely,

τ = L(y) − F (t, y),

where y is the exact solution of (1).

To illustrate the latter, consider the following two examples of numerical methods to solve
the problem
y 0 = f (t, y), (0, T ]
(4)
y(0) = y0 .

Example 1 (Euler forward method) The numerical method reads

yk+1 = yk + hf (tk , yk ).

We see that y 0 is approximated using the first two terms in the Taylor extension y(tk + h) =
y(tk ) + hy 0 (tk ) + · · · , thus,
y(tk + h) − y(tk )
y 0 (tk ) ≈ .
h
In this case F (t, y) = f (t, y).

2
Example 2 (Heun’s method) The numerical method reads

h
yk+1 = yk + (f (tk , yk ) + f (tk+1 , yk + hf (tk , yk ))) .
2
We see here that y 0 has been approximated as in Example 1, however, now F (t, y) =
1
2
(f (tk , yk ) + f (tk+1 , yk + hf (tk , yk ))).

Note: In the simplest case when F (t, y) = f (t, y), as L(y) = f (t, y), the truncation error
can be seen as the difference L(y) − L(y). It turns out that to estimate τ is usually rather
easy, based on some suitable Taylor extension.

However, we are primarily interested in the quality of the approximated solution, thus, in
the difference e(t) = y(t) − z(t). The latter error is referred to as the discretization error
and to estimate it can sometimes be more involved.

Assume that t ∈ I ≡ [t0 , T ] and discretize the interval with a (for simplicity) constant step
h, i.e., we seek the solution y of (1) only at the points tk = t0 + kh for k = 0, 1, 2, · · · n,
t0 + nh = T . From now on the approximated solution z at point tk is denoted by yk , in
contrast to y(tk ), which is the exact solution at tk .

Definition 2 We define the discretization error at time tk as

ek = y(tk ) − yk . (5)

We distinguish between two types of discretization errors - the local and the global dis-
cretization errors.

Definition 3 The local discretization error at time tk+1 is defined as elocal

k+1 = y(tk+1 )−yk+1 ,
provided that at step k we have yk = y(tk ).
The global discretization error at time tk+1 is defined as eglobal
k+1 = y(tk+1 ) − yk+1 , however
yk is the numerical solution at tk and it contains errors from the previous steps.

In other words, the local discretization error is the error we do when we use some numerical
method, when we start with the exact value of the solution at time tk and take one step.
As already noted, we are interested in computing a good approximation of y, it is the
discretization error that is more important than the truncation error. Furthermore, as we
want to compute the approximate solution at some final time T , it is the global discretiza-
tion error that determines the final obtained accuracy.

We derive below estimates for the truncation and discretization errors for the explicit
(forward) and the implicit (backward) Euler methods.

3
Truncation and discretization error for the Euler forward method
The Euler forward (EF) method is defined as follows,

yk+1 = yk + hf (tk , yk ).

Following Definition 1, the truncation error is found to be

Taylor expansion of y(tk+1 )
z }| { from the equation
0 2
y(tk+1 − y(tk ) y(tk ) + hy (tk ) + 0(h ) −y(tk ) z }| {
EF
τk+1 = −f (tk , y(tk )) = − y 0 (tk ) = O(h).
h h
In many cases (and also used in some books), we need the quantity
from Taylor’s theorem, (ii)
z }| { h2
EF
hτk+1 = y(tk+1 ) − y(tk ) − hf (tk , y(tk )) = y(tk+1 ) − y(tk ) − hy 0 (tk ) = y 00 (τ ) = O(h2 ),
2
assuming that the second derivative of y is uniformly bounded.
Next we derive the local discretization error, thus, the error ek+1 = y(tk+1 )−yk+1 , provided
that yk = y(tk ) holds exactly.

h2 00
ek+1 = y(tk+1 ) − yk+1 = y(tk ) + hy 0 (tk ) + y (τ ) − yk − hf (tk , yk ) = hτk+1 = O(h2 ). (6)
2
Thus, we see that the local discretization error for the Euler forward method is smaller
than the truncation error.
The global discretization error en = y(tn ) − yn at the final point of the interval of interest
T is O(h). We use the following result.

Theorem 2 If the local discretization error is O(hp+1 ) then the global discretization is
O(hp ).

The parameter p in Theorem 2 is referred as the order of accuracy. The theorem also
shows that the global error is one order less than the local discretization error.
As the local discretization error for the Euler Forward (EF) method is O(h2 ), then we see
that the global discretization error is O(h). In other words, EF is only first order accurate.

Truncation and discretization error for the Euler backwards method

The Euler backward (EB) method is defined as follows,

yk+1 = yk + hf (tk+1 , yk+1 ).

4
Following Definition 1, analogously to the analysis for EF, the (scaled) truncation error is
found to be
h2
EB
hτk+1 = y(tk+1 )−y(tk )−hf (tk+1 , y(tk+1 )) = y(tk+1 )−y(tk )−hy 0 (tk+1 ) = y 00 (τ ) = O(h2 ),
2
thus, the truncation error is O(h).
Next we derive the local discretization error for EB.
h2 00
ek+1 = y(tk+1 ) − yk+1 = y(tk ) + hy 0 (tk+1 ) + 2
y (τ ) − yk − hf (tk+1 , yk+1 )
= y(tk ) − yk + h (f (tk+1 , y(tk+1 )) − f (tk+1 , yk+1 )) + O(h2 )
= ek + h (f (tk+1 , yk+1 ) + (y(tk+1 ) − yk+1 )fy (tk+1 , η)) + O(h2 )
(7)
Here we use Taylor expansion of f (tk+1 , yk+1 ) around the point (tk+1 , y(tk+1 )), namely,
f (tk+1 , yk+1 ) = f (tk+1 , y(tk+1 ) − (y(tk+1 ) − yk+1 ))
= f (tk+1 , y(tk+1 ) − ek+1 ) = f (tk+1 , y(tk+1 )) − ek+1 fy (tk+1 , η)

We rewrite:
1
(1 − hfy (tk+1 , η))ek+1 = ek + O(h2 ), or, ek+1 = (ek + O(h2 )).
1 − hfy (tk+1 , η)
Since we have assumed that yk = y(tk ), then ek = 0.
The expression |1−hfy (tk+1 , η)| is shown to be larger than 1, see the discussion on stability.
Therefore, we have
 
 1 
 O(h2 )) ≤ O(h2 ).
|ek+1 | ≤ 
1 − hfy (tk+1 , η) 
From the latter inequality we see that the local discretization error for the Euler backward
method is also O(h2 ).
Theorem 2 postulates that the global discretization error is O(h). In other words, EB is
also only first order accurate.

Truncation error for Heun’s method

For those, who are not fed up with Taylor expansions... Note first, that
y 00 = ft + fy y 0 = ft + fy f.
The method yk+1 = yk + h2 (f (tk , yk ) + f (tk+1 , yk + hf (tk , yk )))
Definition 1: y(tk+1h)−y(tk ) − 12 [f (tk , y(tk )) + f (tk+1 , y(tk ) + hf (tk , y(tk )))]
2
y(tk ) + hy 0 + h2 y 00 + O(h3 ) − y(tk )
=
h
− 12 [f (tk , y(tk )) + f (tk , y(tk )) + hft + hf fy + O(h2 )]
= y 0 + h2 y 00 + O(h2 ) − f − h2 (ft + f fy ) + O(h2 ) = O(h2 )

5
Thus, the truncation error τ Heun is O(h2 ). The local discretization error is O(h3 ) and the
global discretization error is then O(h2 ).
The derivation of the local discretization error for Heun’s method is rather technical and
is not included.

References
[1] B. Gustafsson, High Order Difference Methods for Time Dependent PDE, Springer,
2008.

[2] J. Strikwerda, Finite Difference Schemes and Partial Differential Equations, Second
edition, 2004, Madison, Wisconsin.