6 views

Uploaded by agcore

CSCC73 Numerical analysis University of Toronto Richard Pancer 2014

- Mth401 Collection of Old Papers
- Numerical Solutions of Riccati Equation
- Solution Manual for an Introduction to the Finite Element Method 3rd Edition by Reddy
- Class7 - Ode_matlab
- Heaviside
- Laplace transfroms solutions
- Differential Equation
- xpp_doc
- Liapunov direct analysis (lyapunov stability analysis method 1)
- MATH1024 Exam 2008
- stephest algorithm
- ch2 approximation numemrique ( eng version)
- Gravastars
- Paper 1 Reserch Methodology
- Chemical Engineering Mathematics
- Chap_3_ODE
- 45 5795 EE441 20152016 1 1 1 Numerical Integration of the Swing Equaqtion
- HigherOrderLinearDifferentialEquationsUnit-6.pdf
- m.elkaim Nalysis of the Dynamics of a Family System in Terms of Bifurcations
- M2 R08 NovDec 10

You are on page 1of 11

Additional Numerical Methods

Numerical Solution of ODEs

Additional Numerical Methods

Outline

Scientific Computing: An Introductory Survey

Chapter 9 Initial Value Problems for

Ordinary Differential Equations

University of Illinois at Urbana-Champaign

Copyright

for noncommercial, educational use only.

Michael T. Heath

Scientific Computing

Numerical Solution of ODEs

Additional Numerical Methods

1 / 84

Differential Equations

Initial Value Problems

Stability

Differential Equations

2 / 84

Differential Equations

Initial Value Problems

Stability

solution function appearing in ODE

with respect to single independent variable, often

representing time

equivalent first-order system

infinite-dimensional space of functions

first-order ODEs

finite-dimensional approximation

equations

whose solution approximates that of given differential

equation

Michael T. Heath

Scientific Computing

Numerical Solution of ODEs

Additional Numerical Methods

3 / 84

Differential Equations

Initial Value Problems

Stability

4 / 84

Differential Equations

Initial Value Problems

Stability

ODE, since acceleration a is second derivative of position

coordinate, which we denote by y

define k new unknown functions

u1 (t) = y(t), u2 (t) = y (t), . . . , uk (t) = y

Scientific Computing

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

(k1)

(t)

y 00 = F/m

0

u1 (t)

u2 (t)

0

u (t)

u3 (t)

2

..

..

. =

0

u (t)

uk (t)

k1

f (t, u1 , u2 , . . . , uk )

u0k (t)

Michael T. Heath

Scientific Computing

Order of ODE

solution function

Numerical Solution of ODEs

Additional Numerical Methods

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

Defining u1 = y and u2 = y 0 yields equivalent system of

two first-order ODEs

0

u1

u2

=

0

u2

F/m

5 / 84

Differential Equations

Initial Value Problems

Stability

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Example, continued

Scientific Computing

6 / 84

Differential Equations

Initial Value Problems

Stability

General first-order system of ODEs has form

y 0 (t) = f (t, y)

where y : R Rn , f : Rn+1 Rn , and y 0 = dy/dt denotes

derivative with respect to t,

0

y1 (t)

dy1 (t)/dt

y 0 (t) dy2 (t)/dt

2

.. =

..

.

this system

First component of solution u1 is solution y of original

second-order equation

Second component of solution u2 is velocity y 0

yn0 (t)

dyn (t)/dt

function y satisfying ODE

For simplicity, we will often consider special case of single

scalar ODE, n = 1

Michael T. Heath

Scientific Computing

7 / 84

Michael T. Heath

Scientific Computing

8 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Differential Equations

Initial Value Problems

Stability

Numerical Solution of ODEs

Additional Numerical Methods

solution function

y(t0 ) = y0 , which determines unique solution to ODE

solution function at each point, but not actual value y(t) at

any point

time, think of t0 as initial time and y0 as initial value

Hence, this is termed initial value problem, or IVP

provided f is sufficiently smooth

state y0 at time t0 onward, and we seek function y(t) that

describes state of system as function of time

function must be specified at some point t0

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Differential Equations

Initial Value Problems

Stability

Scientific Computing

9 / 84

Differential Equations

Initial Value Problems

Stability

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

10 / 84

Differential Equations

Initial Value Problems

Stability

Family of solutions for ODE y 0 = y

y0 = y

Family of solutions is given by y(t) = c et , where c is any

real constant

Imposing initial condition y(t0 ) = y0 singles out unique

particular solution

For this example, if t0 = 0, then c = y0 , which means that

solution is y(t) = y0 et

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

11 / 84

Differential Equations

Initial Value Problems

Stability

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Stability of Solutions

Scientific Computing

12 / 84

Differential Equations

Initial Value Problems

Stability

Family of solutions for ODE y 0 =

1

2

Solution of ODE is

Stable if solutions resulting from perturbations of initial

value remain close to original solution

Asymptotically stable if solutions resulting from

perturbations converge back to original solution

Unstable if solutions resulting from perturbations diverge

away from original solution without bound

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

13 / 84

Differential Equations

Initial Value Problems

Stability

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

14 / 84

Differential Equations

Initial Value Problems

Stability

Consider scalar ODE y 0 = y, where is constant.

and y(0) = y0 is initial value

For real

> 0: all nonzero solutions grow exponentially, so every

solution is unstable

< 0: all nonzero solutions decay exponentially, so every

solution is not only stable, but asymptotically stable

For complex

Re() > 0: unstable

Re() < 0: asymptotically stable

Re() = 0: stable but not asymptotically stable

Michael T. Heath

Scientific Computing

15 / 84

Michael T. Heath

Scientific Computing

16 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Differential Equations

Initial Value Problems

Stability

Numerical Solution of ODEs

Additional Numerical Methods

Example, continued

coefficients has form

y 0 = Ay

exponentially growing solution components

decaying solution components

corresponding eigenvectors vi , i = 1, . . . , n

P

Express y0 as linear combination y0 = ni=1 i vi

Then

y(t) =

n

X

Differential Equations

Initial Value Problems

Stability

oscillatory solution components

Solutions stable if Re(i ) 0 for every eigenvalue, and

asymptotically stable if Re(i ) < 0 for every eigenvalue, but

unstable if Re(i ) > 0 for any eigenvalue

i vi ei t

i=1

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

17 / 84

Differential Equations

Initial Value Problems

Stability

of solution function at discrete set of points

Numerical solution is generated by simulating behavior of

system governed by ODE

z = Jf (t, y(t)) z

dictated by ODE

conclusions drawn may not be valid globally

Scientific Computing

18 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

be evaluated at any point t

truncated Taylor series, yielding linear ODE

Michael T. Heath

Scientific Computing

determining stability of solutions is more complicated

Numerical Solution of ODEs

Additional Numerical Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

future time t1 = t0 + h for some suitably chosen time

increment h

19 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

20 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Eulers Method

For general system of ODEs y 0 = f (t, y), consider Taylor

series

h2

y(t + h) = y(t) + hy 0 (t) + y 00 (t) +

2

h2

= y(t) + hf (t, y(t)) + y 00 (t) +

2

Eulers method results from dropping terms of second and

higher order to obtain approximate solution value

increments moving across interval in which solution is

sought

In stepping from one discrete point to next, we incur some

error, which means that next approximate solution value

lies on different solution from one we started on

yk+1 = yk + hk f (tk , yk )

Eulers method advances solution by extrapolating along

straight line whose slope is given by f (tk , yk )

whether such errors are magnified or diminished with time

on information at only one point in time to advance to next

point

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

21 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

22 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Example, continued

advance solution from time t0 = 0 to time t1 = t0 + h

step from t1 to t2 = t1 + h = 1.0, obtaining

y2 = y1 + hy1 = 1.5 + (0.5)(1.5) = 2.25

problem at t = 1, y(1) = exp(1) 2.718, but it also differs

from solution through previous point (t1 , y1 ), which has

approximate value 2.473 at t = 1

For example, if t0 = 0, y0 = 1, and h = 0.5, then y1 = 1.5,

whereas exact solution for this initial value is

y(0.5) = exp(0.5) 1.649

Michael T. Heath

Scientific Computing

23 / 84

Michael T. Heath

Scientific Computing

24 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Example, continued

Example, continued

with time

< interactive example >

time

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

25 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

26 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Global error : difference between computed solution and

true solution y(t) passing through initial point (t0 , y0 )

floating-point arithmetic

ek = yk y(tk )

approximation method used and would remain even if all

arithmetic were exact

`k = yk uk1 (tk )

accuracy of numerical solutions of ODEs, so we will

henceforth ignore rounding error

Scientific Computing

Scientific Computing

types of error

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

point (tk1 , yk1 )

27 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

28 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Global error is generally greater than sum of local errors if

solutions are unstable, but may be less than sum if

solutions are stable

Having small global error is what we want, but we can

control only local error directly

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

29 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

30 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Order of Accuracy

`k = O(hp+1

k )

Then local error per unit step, `k /hk = O(hpk )

Under reasonable conditions, ek = O(hp ), where h is

average step size

< interactive example >

Michael T. Heath

Scientific Computing

31 / 84

Michael T. Heath

Scientific Computing

32 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Stability

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

numerical method is to apply it to scalar ODE y 0 = y,

where is (possibly complex) constant

cause resulting numerical solutions to diverge from each

other without bound

initial condition

by instability of solution to ODE, but can also be due to

numerical method itself, even when solutions to ODE are

stable

growth of numerical solution

Determine accuracy of numerical method by comparing

exact and numerical solutions

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

33 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

34 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

If is real, then h must lie in interval (2, 0), so for < 0,

we must have

2

h

which means that

yk = (1 + h)k y0

If Re() < 0, exact solution decays to zero as t increases,

as does computed solution if

eh = 1 + h +

|1 + h| < 1

which holds if h lies inside circle in complex plane of

radius 1 centered at 1

(h)2 (h)3

+

+

2

6

first-order accurate

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

35 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

36 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

series

y(t + h) = y(t) + hy 0 (t) + O(h2 )

If there were no prior errors, then we would have

yk = y(tk ), and differences in brackets on right side would

be zero, leaving only O(h2k ) term, which is local error

If we take t = tk and h = hk , we obtain

Subtracting this from Eulers method,

ek+1 = yk+1 y(tk+1 )

= [yk y(tk )] + hk [f (tk , yk ) f (tk , y(tk ))] O(h2k )

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

37 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

38 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

and propagated error

From Mean Value Theorem,

f (tk , yk ) f (tk , y(tk )) = Jf (tk , )(yk y(tk ))

of f with respect to y

So we can express global error at step k + 1 as

Step size h

Jacobian Jf , which is determined by particular ODE

Thus, global error is multiplied at each step by growth

factor I + hk Jf

Errors do not grow if spectral radius (I + hk Jf ) 1,

which holds if all eigenvalues of hk Jf lie inside circle in

complex plane of radius 1 centered at 1

Michael T. Heath

Scientific Computing

39 / 84

Michael T. Heath

Scientific Computing

40 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

ODE, we want to take large steps to reduce computational

cost, but must also take into account both stability and

accuracy

difference quotient

stability restrictions

y 00

desired accuracy is achieved

difference between results obtained using methods of

different orders or different step sizes

approximately (h2k /2)y 00 , so choose step size to satisfy

p

hk 2 tol /ky 00 k

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

0

yk0 yk1

tk tk1

41 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Implicit Methods

Scientific Computing

42 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

time tk to advance solution to time tk+1

determine yk+1

limited stability region

method or fixed-point iteration to solve for yk+1

at time tk+1 , which makes method implicit

explicit method, such as Eulers method, or from solution at

previous time step

yk+1 = yk + hk f (tk+1 , yk+1 )

argument yk+1 before we know its value

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

43 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Consider nonlinear scalar ODE

condition y(0) = 1

y0

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

44 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

y 3

with initial

obtain implicit equation

Given extra trouble and computation in using implicit

method, one might wonder why we bother

y1 = y0 + hf (t1 , y1 ) = 1 0.5y13

for solution value at next step

larger stability region than comparable explicit methods

fixed-point iteration or Newtons method

To obtain starting guess for y1 , we could use previous

solution value, y0 = 1, or we could use explicit method,

such as Eulers method, which gives y1 = y0 0.5y03 = 0.5

Iterations eventually converge to final value y1 0.7709

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

45 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Scientific Computing

46 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Growth factor

scalar ODE y 0 = y, obtaining

1

= 1 + h + (h)2 +

1 h

yk+1 = yk + hyk+1

(1 h)yk+1 = yk

k

1

yk =

y0

1 h

Thus, for backward Euler to be stable we must have

1

1 h 1

backward Euler method is first-order accurate

Growth factor of backward Euler method for general

system of ODEs y 0 = f (t, y) is (I hJf )1 , whose

spectral radius is less than 1 provided all eigenvalues of

hJf lie outside circle in complex plane of radius 1 centered

at 1

So stability region for backward Euler method includes

entire left half of complex plane, or interval (, 0) if is

real

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

of ODEs is entire left half of complex plane

47 / 84

Michael T. Heath

Scientific Computing

48 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Trapezoid Method

Higher-order accuracy can be achieved by averaging Euler

and backward Euler methods to obtain implicit trapezoid

method

stable for any positive step size, which means that it is

unconditionally stable

desired accuracy is only constraint on choice of step size

ODE y 0 = y, obtaining

explicit method of comparable order and attain much

higher overall efficiency despite requiring more

computation per step

1 + h/2 k

yk =

y0

1 h/2

Method is stable if

1 + h/2

1 h/2 < 1

its accuracy is only of first order, which severely limits its

usefulness

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

49 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

50 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Its growth factor

!

2 3

h

h

h

h

1 + h/2

=

1+

1+

+

+

+

1 h/2

2

2

2

2

= 1 + h +

(h)2 (h)3

+

+

2

4

than explicit methods, but allowable step size is not always

unlimited

trapezoid method is second-order accurate

method has growth factor (I + 12 hJf )(I 12 hJf )1 , whose

spectral radius is less than 1 provided eigenvalues of hJf

lie in left half of complex plane

< interactive example >

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

are unconditionally stable, but not all implicit methods have

this property

51 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

52 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

this has favorable property of damping errors in numerical

solution

components have disparate time scales or whose time

scale is small compared to interval over which it is studied

of different type may arise

Jacobian matrix Jf differ greatly in magnitude

large negative real parts, corresponding to strongly damped

components of solution, or

large imaginary parts, corresponding to rapidly oscillating

components of solution

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

53 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

54 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

because rapidly varying component of solution forces very

small step sizes to maintain stability

of solution, but accuracy restriction depends on slowly

varying component, so step size may be more severely

restricted by stability than by required accuracy

solution satisfying initial condition is y(t) = 1 + t

(i.e., c = 0)

solving stiff ODEs because of severe stability limitation on

step size

exact for this problem

of its unconditional stability

arithmetic, let us perturb initial value slightly

provided suitable method is chosen

Michael T. Heath

Scientific Computing

Scientific Computing

55 / 84

Michael T. Heath

Scientific Computing

56 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Example, continued

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Example, continued

With step size h = 0.1, first few steps for given initial values

are

t

0.0

0.1

0.2

0.3

0.4

exact sol. 1.00 1.10 1.20

1.30

1.40

Euler sol. 0.99 1.19 0.39

8.59 64.2

Euler sol. 1.01 1.01 2.01 5.99

67.0

point, and resulting large value causes numerical solution

to diverge radically from desired solution

Jacobian for this ODE is Jf = 100, so stability condition

for Eulers method requires step size h < 0.02, which we

are violating

each tiny perturbation results in wildly different solution

Any point deviating from desired particular solution, even

by only small amount, lies on different solution, for which

c 6= 0, and therefore rapid transient of general solution is

present

< interactive example >

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

57 / 84

Michael T. Heath

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Numerical Solution of ODEs

Additional Numerical Methods

Example, continued

Scientific Computing

58 / 84

Eulers Method

Accuracy and Stability

Implicit Methods and Stiffness

Example, continued

problem and is extremely insensitive to initial value

t

0.0

0.1

0.2

0.3

0.4

exact sol. 1.00 1.10 1.20 1.30 1.40

BE sol.

0.00 1.01 1.19 1.30 1.40

BE sol.

2.00 1.19 1.21 1.30 1.40

Even with very large perturbation in initial value, by using

derivative at next point rather than current point, transient

is quickly damped out and backward Euler solution

converges to desired solution after only few steps

backward Euler method for stable solutions

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

59 / 84

Michael T. Heath

Single-Step Methods

Extrapolation Methods

Multistep Methods

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

60 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

Eulers method can be derived from Taylor series

expansion

which are of one of following types

higher-order single-step methods

Taylor series

Runge-Kutta

Extrapolation

y(t + h) = y(t) + h y 0 (t) +

Multistep

h2 00

h3 000

y (t) +

y (t) +

2

6

Multivalue

We briefly consider each of these types of methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

yk+1 = yk + hk yk0 +

61 / 84

Numerical Solution of ODEs

Additional Numerical Methods

62 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

motivation to Taylor series methods, but they do not require

computation of higher derivatives

y 00 = ft (t, y) + fy (t, y) y 0

derivatives by evaluating f several times between tk and

tk+1

where subscripts indicate partial derivatives with respect to

given variable

become too complicated to be practical to compute, so

Taylor series methods of higher order have not often been

used in practice

yk+1 = yk +

hk

(k1 + k2 )

2

where

k1 = f (tk , yk )

Scientific Computing

Scientific Computing

Runge-Kutta Methods

of y, which can be obtained by differentiating y 0 = f (t, y)

using chain rule, e.g.,

Michael T. Heath

Michael T. Heath

Single-Step Methods

Extrapolation Methods

Multistep Methods

h2k 00

y

2 k

k2 = f (tk + hk , yk + hk k1 )

63 / 84

Michael T. Heath

Scientific Computing

64 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

but remains explicit by using Euler prediction yk + hk k1

instead of y(tk+1 ) in evaluating f at tk+1

history of solution prior to time tk , which makes them

self-starting at beginning of integration, and also makes it

easy to change step size during integration

scheme

hk

(k1 + 2k2 + 2k3 + k4 )

yk+1 = yk +

6

where

easy to program, which accounts in part for their popularity

k1 = f (tk , yk )

error estimate on which to base choice of step size

k3 = f (tk + hk /2, yk + (hk /2)k2 )

k4 = f (tk + hk , yk + hk k3 )

< interactive example >

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

65 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

Extrapolation Methods

uses six function evaluations per step to produce both

fifth-order and fourth-order estimates of solution, whose

difference provides estimate for local error

method to integrate ODE over given interval tk t tk+1

using several different step sizes hi , and yielding results

denoted by Y (hi )

Dormand and Prince

Y (0) = y(tk+1 )

Interpolating polynomial or rational function Y (h) is fit to

these data, and Y (0) is then taken as approximation to

Y (0)

that are effective for many problems, but which are

relatively inefficient for stiff problems or when very high

accuracy is required

accuracy, but they are much less efficient and less flexible

than other methods for ODEs, so they are not often used

unless extremely high accuracy is required and cost is not

significant factor

< interactive example >

methods with superior stability properties that are suitable

for solving stiff ODEs

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

67 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Multistep Methods

Scientific Computing

Starting guess is conveniently supplied by explicit method,

so the two are used as predictor-corrector pair

iteration) until some convergence tolerance is met, but it

may not be worth expense

yk+1 =

68 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

previous point to estimate solution at next point

m

X

66 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

i yk+1i + h

i=1

m

X

i f (tk+1i , yk+1i )

i=0

used instead, giving PECE scheme (predict, evaluate,

correct, evaluate)

interpolation

If 0 = 0, method is explicit, but if 0 6= 0, method is implicit

evaluation of f in PECE scheme yields improved value of

0

yk+1

for future use

explicit methods, but require starting guess for yk+1

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

69 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

70 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

h

0

)

yk+1 = yk + (3yk0 yk1

2

Simplest second-order accurate implicit method is

trapezoid method

h 0

yk+1 = yk + (yk+1

+ yk0 )

2

One of most popular pairs of multistep methods is explicit

fourth-order Adams-Bashforth predictor

h

0

0

0

yk+1 = yk + (55yk0 59yk1

+ 37yk2

9yk3

)

24

and implicit fourth-order Adams-Moulton corrector

h

0

0

0

yk+1 = yk + (9yk+1

+ 19yk0 5yk1

+ yk2

)

24

using an implicit multistep method designed for stiff ODEs,

as fixed-point iteration fails to converge for reasonable step

sizes

< interactive example >

Scientific Computing

Scientific Computing

multistep method can be solved by Newtons method or

similar method, again with starting guess supplied by

solution at previous step or by explicit multistep method

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

71 / 84

Michael T. Heath

Scientific Computing

72 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

Stability and accuracy of some Adams methods are

summarized below

Stability threshold indicates left endpoint of stability interval

for scalar ODE

Error constant indicates coefficient of hp+1 term in local

truncation error, where p is order of method

family of implicit multistep methods

BDF methods, typified by popular formula

yk+1 =

Explicit Methods

Stability

Error

Order threshold constant

1

2

1/2

2

1

5/12

3

6/11

3/8

4

3/10

251/720

1

6h 0

(18yk 9yk1 + 2yk2 ) +

y

11

11 k+1

< interactive example >

Implicit Methods

Stability

Error

Order threshold constant

1

1/2

2

1/12

3

6

1/24

4

3

19/720

than corresponding explicit methods of same order

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

73 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

complicated way

methods, they are still not necessarily unconditionally

stable

difference between predictor and corrector

unconditionally stable, even if it is implicit

Being based on interpolation, they can efficiently provide

solution values at output points other than integration

points

Scientific Computing

74 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

explicit methods, but must be iterated to convergence to

enjoy this benefit fully

formulas are most conveniently based on equally spaced

intervals for several consecutive points

Michael T. Heath

Scientific Computing

yk are needed initially

Numerical Solution of ODEs

Additional Numerical Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

effective for solving stiff ODEs

75 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Multivalue Methods

Scientific Computing

76 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

since past history of solution is most easily maintained at

equally spaced intervals

representing interpolating polynomial so that parameters

are values of solution and its derivatives at tk

based on polynomial interpolation, but avoid some

implementation difficulties associated with multistep

methods

Lagrange, representation of polynomial

Solution is advanced in time by simple transformation of

representation from one point to next, and it is easy to

change step size

that interpolating polynomial itself can be evaluated at any

point, not just at equally spaced intervals

multistep methods but are more convenient and flexible to

implement, so most modern software implementations are

based on them

of representation as linear combination of successive

solution and derivative values with fixed weights

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

77 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

78 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

By differentiating Taylor series

y(tk + h) = y(tk ) + hy 0 +

y = f (t, y)

tk+1 = tk + h are given approximately by transformation

value at four different points, we represent it by its value

and first three derivatives at single point tk

yk

hy 0

k

yk =

(h2 /2)y 00

k

3

000

(h /6)yk

Michael T. Heath

Scientific Computing

h2 00 h3 000

y + yk +

2

6

y

k+1 = Byk

where matrix B is given by

1

0

B=

0

0

79 / 84

Michael T. Heath

1

1

0

0

1

2

1

0

1

3

3

1

Scientific Computing

80 / 84

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

Numerical Solution of ODEs

Additional Numerical Methods

Single-Step Methods

Extrapolation Methods

Multistep Methods

Example, continued

term to foregoing prediction to obtain final value

yk+1 = y

k+1 + r

r=

0

yk+1

)

3

8

3

4

1 T

6

method given earlier

r2 = 1

Remaining three components of r can be chosen in

various ways, resulting in different methods, analogous to

different choices of parameters in multistep methods

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

Scientific Computing

81 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

82 / 84

Single-Step Methods

Extrapolation Methods

Multistep Methods

routines: they automatically adapt to given problem,

varying order and step size of integration method as

necessary to meet user-supplied error tolerance efficiently

changing components of r

nonstiff problems, and some even detect stiffness

automatically and select appropriate method accordingly

and strategies for deciding when to change order and step

size, have led to development of powerful and efficient

software packages for solving ODEs based on

variable-order/variable-step methods

Scientific Computing

Scientific Computing

Variable-Order/Variable-Step Solvers

need merely rescale components of yk to reflect new step

size

Michael T. Heath

Ordinary Differential Equations

Numerical Solution of ODEs

Additional Numerical Methods

special starting procedures: one can simply start with

first-order method, which requires no additional starting

values, and let automatic order/step size selection

procedure increase order as needed for required accuracy

83 / 84

Michael T. Heath

Scientific Computing

84 / 84

- Mth401 Collection of Old PapersUploaded bycs619finalproject.com
- Numerical Solutions of Riccati EquationUploaded byAliff Ikmal
- Solution Manual for an Introduction to the Finite Element Method 3rd Edition by ReddyUploaded bya493332442
- Class7 - Ode_matlabUploaded byMohd Fauzi Zanil
- HeavisideUploaded byBali Amin
- Laplace transfroms solutionsUploaded bylarryfole
- Differential EquationUploaded byNadda Muhamad
- xpp_docUploaded byCristian Sandoval
- Liapunov direct analysis (lyapunov stability analysis method 1)Uploaded byOdera Tinman Nwagba
- MATH1024 Exam 2008Uploaded bymarana77
- stephest algorithmUploaded bydah7542gmail
- ch2 approximation numemrique ( eng version)Uploaded byAouiti Abdelkarim
- GravastarsUploaded bygatitahanyou
- Paper 1 Reserch MethodologyUploaded byamanpanghal
- Chemical Engineering MathematicsUploaded byfarukh jamil
- Chap_3_ODEUploaded byFred H Halder
- 45 5795 EE441 20152016 1 1 1 Numerical Integration of the Swing EquaqtionUploaded byAlluri Appa Rao
- HigherOrderLinearDifferentialEquationsUnit-6.pdfUploaded byCrazy Hacker
- m.elkaim Nalysis of the Dynamics of a Family System in Terms of BifurcationsUploaded byDrace Klimt
- M2 R08 NovDec 10Uploaded byOwthaman Rooben
- General Eng StandardsUploaded byPageduesca Rouel
- 05 Stability AnalysisUploaded byalareeqi
- asme-interpack05Uploaded bySaher Saher
- 0004Uploaded bypeanuts22
- pdematUploaded bySounak Sadhukhan
- Development, Optimization, and Analysis of Cellular Automaton Algorithms to Solve Stochastic Partial Differential EquationsUploaded byIRJET Journal
- tareai1Uploaded byEri Ka
- 07-Lyapunov Stability Theory(1)Uploaded byvhiga gionata
- locs_hk_rs_c4.pdfUploaded by何家銘
- Cosc3x11 Scp Outline 2014Uploaded byaucyaucy123

- MGTC70Uploaded byagcore
- Chap13_8up- Random Numbers and SimulationUploaded byagcore
- Chap12_8up- Fast Fourier TransformUploaded byagcore
- Chap11_8up- Partial Differential EquationsUploaded byagcore
- Chap10_8up- Boundary Value Problems for ODEsUploaded byagcore
- Chap08_8up- Numerical Integration and DifferentiationUploaded byagcore
- Chap07 8up- InterpolationUploaded byagcore
- chap06_8up- OptimizationUploaded byagcore
- Chap05_8up- Nonlinear EquationsUploaded byagcore
- Chap04_8up- Eigenvalue ProblemsUploaded byagcore
- Chap03_8up- Linear Least SquaresUploaded byagcore
- Chap02_8up- Systems of Linear EquationsUploaded byagcore
- Chap01_8up- Scientific ComputingUploaded byagcore

- MAR6862 Customer Relationship Management Syllabus Sect ALL, Cooke, AUploaded byAnindita A. Primastya
- Halo: Glasslands; GlasslandsUploaded byMacmillan Publishers
- PanoramicPanda's P.T.U. Tools-1.pdfUploaded byHarry Cross
- Thyroid GlandUploaded byXandy Jaraba
- [IJIT-V3I2P10]: T. Kalaiselvi, P. Nagaraja, Z. Abdul BasithUploaded byIJITJournals
- PHY 1 Mod 09 Mod(Collision)Uploaded byShamim Imtiaz
- Herbal Remedies for Treatment of HypertensionUploaded byIan Dale
- Ti4443qUploaded byMarcM.Cataluňa
- Inwa.txtUploaded bydfgrtg454g
- Detailed SSC CGL Syllabus (Tier I) - Based on New Pattern 2017 - Testbook Blog.pdfUploaded byKabir Sai Prasanna Suryavansh
- Beyond the Diagnostic Trouble TreeUploaded byFabricio Lima
- machagija.docxUploaded byshilla benson
- Clerking TemplateUploaded byNur Atiqah Mohd Azli
- SEC ISD Requirement for 380 KV SS 4 (Shedgum)Uploaded bymajeeth_aneesa
- Federigo's FalconUploaded byStephen Lloyd Gerona
- Instrument System - Week 2Uploaded byMuhammad Ismunandar Al Fajr
- 014 Duke Elder Examination Information for Candidates 2013Uploaded byPartialCharge
- English Translation 19 - Kaaba, Baitullah, Masjid Al-Haraam - Expanded Version of Translation 13Uploaded byi360.pk
- Exam Cram70662 FinalUploaded byShanell TheBeliever Swinton
- Killer Press Kit SamplesUploaded byBruce Petherick
- Bits and Pieces From Behind the ScenesUploaded byMaggador Edward Alexander
- Radhanath Ray-Oriya PoetUploaded bySanjay Chandwani
- Epidemiology, Risk Factors, Pathogenesis, And Natural History of Thoracic Aortic AneurysmUploaded byNathaliazuos
- Ass. About RizalUploaded byrochielanciola
- Husserl - Cartesian MeditationsUploaded byRoland Adorjani
- gangguan juwa bunuh diri.pdfUploaded bydanang
- spanish speaking country research projectUploaded byapi-264002455
- CSB Nitrogen Asphyxiation BulletinUploaded byNguyễn Ngọc Sơn
- parent guide 2018 19Uploaded byapi-236427965
- Premillennialism in Revelation 20:1-6Uploaded byJay Smith