CHE 555

NUMERICAL METHODS
AND OPTIMIZATION

WEEK 1

Introduction to Numerical Methods

Mathematical modeling
Approximation and round off errors
Truncation errors and Taylor Series

LESSON OUTCOMES
At the end of this topic, the students will be able:

To describe numerical techniques as compared

to analytical methods

To use Taylor series expansion to approximate

a function

To perform error analysis associated with

numerical methods

Why Numerical Method?

Could handle large systems of equations,
nonlinearity and complex geometries that is not
common.
It provide approximate solutions to many of the
engineering problems.

What is Numerical Method_cont

Powerful analysis tool in problem solving and
understanding problem in mathematical language
Techniques by which mathematical problems are
formulated, so that they can be solved with
arithmetic operations
The role of numerical method in solving engineering
problem:
PROBLEM
FORMULATION
Fundamental laws are used
to develop mathematical
equations that can represent
the specific problem

INTERPRETATION

SOLUTION
Suitable numerical methods
are then selected to solve
the mathematical equations

The results obtained can

then be used to
predict/analyze/understand
the specific problem better
5

 Mathematical modeling
Is the use of mathematics to

Describe real world phenomena

Investigate important questions about the observed world
Explain real world phenomena
Test ideas
Make predictions about the real world

The real world refers to

Engineering
Physics
Physiology
Ecology
Wildlife management Chemistry
Economics
Sports
Etc

 A mathematical model is represented as a functional

relationship of the form

 Dependent variable
Observed behaviour/state/phenomenon of a system
Characteristic that reflects behaviour or state of the system
i.e. y, f(x), f(t)

Independent variable
Dimension that determine a system i.e. time, t , x

Parameter
Quantity that serves to relate to functions and variables
Reflective of the systems properties or composition (ex: T, P, pH)

Forcing functions
External influence that acts on system i.e. acceleration gravity, g
8

Example

Apply Newtons second law,

And also can write as

dx
dt

F = ma

dv
F
dt
or

d 2x
m 2 F
dt

Eq. relates a linear position x

(dependent variable) to the applied
force, F (forcing function) and the
time, t (independent variable). The
mass, m is the only parameter in
the above model.
9

Example of mathematical modeling

10

Example
Assume that interested to predict the velocity of the falling parachutist with time

Use fundamental knowledge to find a

mathematical equation correlates the
velocity to the various forces acting on the
parachutist
Newtons 2nd law of Motion
the time rate change of momentum of a body
is equal to the resulting force acting on it.
The model is formulated as
F = ma
F=net force acting on the body (N)
m=mass of the object (kg)
a=its acceleration (m/s2)

11

acceleration a
dv
dt

dv
dt
F FD FU

FD downward forceof gravity

FU upward forceof air resistance

FD m g
FU cv
Model relates acceleration of falling
object to the forces acting on it,

dv
dt

mg cv
m

c
m

(differential equation)

12

Exact or analytical solution: it exactly satisfies the original equation

dependent variable

forcing functions

t,s

v,(m/s)

0.00

16.40

27.77

35.64

41.10

10

44.87

12

47.87

independent variable

parameter

Analytical solution of the parachutist

13

53.39

Unfortunately, there are many mathematical

models that cannot be solved exactly.

Numerical solution that approximates the exact solution

14

dv
dt
dv
dt
v( t

m g cv

m
v
t

i 1 )

v( t

v( t )
i

ti 1 ti

i 1 )

c
m

v( t )
i

ti 1 ti
g

c
m

v(t )
i

v(ti1 ) v( ti ) g v(ti ) (ti 1 ti )

m

The use of finite difference to approximate the

first derivative of v with respect to t

Approximate or
numerical solution
15

Numerical solution

t,s

v,(m/s)

0.00

19.60

32.00

39.85

44.82

10

47.97

12

49.96

53.39

v(ti1 )

v( ti ) g v( ti ) (ti 1 ti )
m

Comparison between the exact and

numerical solution

16

Approximation and Roundoff Errors

Significant figures
Numbers to be used in confidence

98

2 significant figures

98.09

4 significant figures

0.0098

2 significant figures
17

Important of significance figures in numerical methods:

Numerical methods yield approximate results, therefore,
need to develop criteria to specify the confident in
approximate result.
Although quantities such as , e, or 7 represent specific
quantities, they cannot be expressed exactly by a limited
number of digits. Computers retain only a finite number of
significant figures.
= 3.141592653589793238462643

18

 Accuracy and Precision

Increasing precision

How closely a computed

or measured value agree
Increasing accuracy with the true value

How closely individual

computed or measured
values agree with each
other

Inaccurate & imprecise

accurate & imprecise

Inaccurate & precise

accurate & precise

19

 Error definitions
Error

Truncation errors ~ result

when approximations are
used to represent exact
mathematical procedures
Round-off errors ~ result
when numbers having a
limited significant figures
are used to represent
exact numbers

True value

Approximation
value
True value = Error + Approximation value

20

True value = error + approximation value

(1)

True error (Et)= true value approximation

(2)

(3)

t designates true percent relative error

21

Calculation of errors
True value of length of a bridge is 10,000 cm.
When you measure, the length recorded is
9,999 cm. Compute the true error and true
percentage relative error of the bridge.

Answer: 1 cm and 0.01%

22

However, in actual situation, true value is rarely available.

Therefore, need to estimate the true value approximation
(4)

In numerical method, iterative approach is used to

compute answer, in which error is estimated as the
difference between previous and current approximations.
(5)

The signs of error can be negative or positive,

Absolute value of error, |a| need to be lower than
prespecified percent tolerance, s
n is significant figures.
23

Error estimates for iterative method

Suppose that we have exponential function as,
Starting with the simplest version, ex=1, add
terms to estimate e0.5. Compute true (t) and
approximate error (a) after each term is added
until |a| falls below s , conforming to 3
significant figures. Note that true value of e0.5
is 1.648721.
24

 Answer

25

 Round-off errors

Results when numbers having limited significant

figures are used to represent exact numbers

ln 2 = 0.693 147 180 559 945 309 41...

A device only shows 8
significant numbers, so
round-off error is discrepancy
introduced by omission of
significant figures.
Round-off error for this case is
0.00000000055994530941
26

 Other example of roundoff error

27

 Truncation errors and Taylor series

Truncation errors
Truncation error is the discrepancy introduced by the fact that
numerical methods may employ approximations to represent
exact mathematical operations and quantities.
Truncation error are errors resulted from using an approximation
in place of an exact mathematical procedure.

The difference between the calculated value using exact

mathematical equation and approximation mathematical
equation.

28

 Taylor series
Provides a means to predict a function value at one point in terms of the
function value and its derivative at another point

Zero order
First order
Second
order

nth order

29

Taylor series by defining a step size h = xi+1 - xi

(6)

f ( n 1) ( ) n 1
Rn
h
(7)
(n 1)!

30

31

 Remainder for the Taylor series Expansion

Remainder term, Rn, accounts for all terms from (n+1) to infinity.
f ( n 1) ( ) n 1
Rn
h
( n 1)!

It also usually expressed as:

Rn O (h n 1 )

Let say we truncated the Taylor series expansion after zero-order term to
yield
f ( xi1 ) f ( xi )
Remainder term, Rn for zero order version

R0 f '( xi ) h

f ' '( xi )

3
f
( xi )
h2
h 3 ...
2!
3!

Let truncate the remainder itself,

R0 f '( xi ) h
This result is still inexact because neglected second and higher order terms.
32

Remainder for the Taylor series Expansion

33

 Alternative simplification that tranforms the approximation into an

equivalence based on graphical insight
derivative mean-value theorem states that if a function f(x) and its
derivative are continous over interval from xi to xi+1,
there exist at least one point on the function that has a slope, designated by
f(), parallel to line joining f (xi) and f(xi+1)
Thus,
So,
Zero order version

First order version

34

 Numerical
differentiation
Forward finite divided difference

Backward finite divided difference

Centered finite divided difference

35

Forward finite divided difference approximation of first derivative

(14)

Where,

(15)

Backward finite divided difference approximation of first derivative

f(xi ) f(xi 1 ) R1

h
h
f
f'(xi ) i O(h)
h
f'(xi )

Where,

(16)

h xi xi 1

Centered finite divided difference approximation of first derivative

(17)
Where,

h xi 1 xi xi xi 1

36

Example (Example 4.4 (text book))

Use forward and backward difference approximations of O(h) and a centered
difference approximation of O(h2) to estimate the first derivative of

f(xi ) 0.1x 4 0.15 x 3 0.5 x 2 0.25 x 1.2

at x = 0.5 using a step size h = 0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.

37

Exercise
Use forward and backward and a centered difference to estimate the first
derivative of the function

f(xi ) 0.5 x 3 0.1x 2 0.8 x 0.7

at x = 0.5 using a step size h=0.5. Repeat using h = 0.25. Also calculate the
true percent relative error for each approximation.
Ans:
h=0.5
FDM:1.525
41.9%
BDM:0.875 18.60%
CDM:1.200 11.63%
h=0.25
FDM:1.26875
18.02%
BDM:0.94375
12.21%
CDM:1.10625
2.91%
38

 Higher derivatives

Second forward finite difference approximation of higher derivatives

Second backward finite difference approximation of higher derivatives

Second centred finite difference approximation of higher derivatives

39

 Error propagation
This section is to study how errors in numbers can
propagate through mathematical functions. If we multiply
two numbers that have errors, we would like to estimate
the error in the product.
If a function f is dependent on
(a) a single independent variable x : f(x)
(b) two independent variables x and y : f(x, y)
(c) several independent variables x1, x2, x3, ... ,xn : f(x1 , x2 ,..., xn ).

40

Function of a single variable.

Let x be the true value and
x* be an approximate value of x
Then, TSE for f(x) computed near f(x*) is given by

f ' ' ( x*)

f(x) f(x*) f'(x * )( x x*)
( x x*) 2 ...
2
Truncating after the first derivative term and rearranging the remaining terms
to give

f(x) f(x*) f ' (x*)( x x*)

where

(21)

f(x*) f ' (x*) x *

f ' (x*) f ( x) f ( x*) is an estimate of the error of the function

x* x x * is an estimate of the error of independen t variable x
Eq.(21) provides 2 capabilities:
1. to approximate the error in f(x) knowing its derivative.
2. to approximate the error in the independent variable x.
41

Example
Given a value of x* = 2.5 with an error of x* = 0.01, estimate the
resulting error in the function, f(x)=x3.
Solution

f(x*) f ' (x*) x *

So,
f(x*) 3(2.5) 2 (0.01) 0.1875
Because f (2.5) 15.625, it can predict th at
f( 2.5 ) 15.625 0.1875
Or the true value lies between 15.4375 and 15.8125. In fact, if x ~2.49,
f(x) could be 15.4382 and if x ~ 2.51, it would be 15.8132.
The first order error analysis provides a fairly close estimate of the true error.
42

Exercise
Knowing a value of x* = 2.0 with an error of x* = 0.01, estimate the resulting
error in the function

f(x) = 0.5x3-0.1x2+0.8x-0.7

Ans: f(2.0)=4.5 0.064

43

Function of a more than One variable.

(22)

(23)

Refer section 4.2.2

for examples

44

 Relative error
Refer section 4.2.3
for example

Condition number

45

 Condition no equals 1 indicates that functions

relative error is identical to the relative error in
x
Condition no greater than 1 indicates relative
error is amplified.
Condition no less than 1 indicates relative
error is attenuated.
Function with very large values are said to be
ill-conditioned.
46

 Total Numerical Error

Total numerical errors = truncation error + round off error
Roundoff error by increase no. of significant figures or
reduce no. of computation in analysis
Truncation error by decreasing step size (h) or increase
no. of computation in analysis

47

Control numerical error

avoid subtract 2 nearly equal numbers to
avoid loss of significance
Use Taylor series for truncation and roundoff
error analysis
Perform numerical experiments
- repeat computation with different step size or method
and compare results

48