MA 523: Numerical Analysis

Instructor: Rajendra Kr. Ray

Credits: 3-1-0-4

Monday: 8:00 8:50 (Lecture)

Tuesday: 14:00 14:50 (Lecture)
Wednesday: 11:00 11:50 (Lecture)
Friday: 10:00 10:50 (Tutorial)
1
 Evaluation Pattern

Two Quizzes -- 40 Final Exam -- 50

Tutorial -- 10

Institute requirement: 70% attendance

2
 References:

Texts Books:
K. E. Atkinson, An Introduction to Numerical Analysis, 2nd Edition, John Wiley, 2008.
R. L. Burden and J. D. Faires, Numerical Analysis, 7th ed., Thomson Learning, 2001.

References:
M. T. Heath, Scientific Computing: An Introductory Survey, McGraw Hill, 2002.
S. D. Conte and Carl de Boor, Elementary Numerical Analysis, McGraw Hill, 1988.
A. Ralston and P. Rabinowitz, A First Course in Numerical Analysis, Dover Publications,
2001.
Brian Bradie, A friendly introduction to Numerical Analysis, Pearson Education, 2007.

3
 Approximate Lecture Distributions:
Lecture -1 & 2: Introduction to Numerical Computing, Significant digits, Sensitivity and conditioning .

Lecture -3 & 4: Approximations and Errors in Computing , Stability and accuracy

Lecture -5 - 9: Roots of Nonlinear Equations (Bisection Method, Newtons methods and its variants ,
Secant method, Fixed-point iteration method, etc. , order of convergence)

Lecture -10 & 16: Existence and uniqueness of interpolating polynomial, Lagrange polynomials, Cubic
spline, Least-Square approximations.

Lecture -17 & 27: System of Linear Equations: Direct Solution process (Gauss Elimination, Partial -
Pivoting, etc.), Iterative Solution process (Gauss-Seidel, SOR, QR decomposition, etc.),

Lecture -28 & 32: Numerical Differentiation and Integration: Forward, Backward and Central Difference,
Newton-Cotes integration formulae, Composite rules, Gaussian quadrature, etc.

Lecture -33 & 42: Initial and Boundary Value Problem (IVP & BVP): Taylor series method, Euler
methods, Runge-Kutta methods, etc., Finite difference methods and Shooting methods, etc.

Tutorials: Self Learning (SL); Implementation of studied methods, by hand calculations, through
computer programming. 4
 Unit - 1 : Introduction to Numerical Computing

What is Numerical Computing ?

It is an approach for solving complex mathematical problems using
only simple arithmetic operations

Numerical Computations:

Play an indispensable role in solving real life problems

Have been in use even before digital computers appeared

The arrival of digital computers has, however, enhanced the speed and accuracy of
numerical computations.

Numerical Analysis is the combinations of Mathematical study of the numerical methods

with numerical Computing.
5
6
7
8
 Characteristics of Numerical Computing

Accuracy : Every method of numerical computing introduces errors

Truncation errors
Roundoff errors

Rate of Convergence : Many numerical methods are based on the idea of an

Iterative process

Numerical Stability : Errors introduced into computation, from whatever

source, propagate in different ways

Efficiency : The amount of effort required by both human and computer

to implement the method

Mathematical Methods
Numerical Computing

Computers
9
(1662)

(3000 BC.)

(1642)

10
Punched card loom by Jacquard(1801)

11
Difference Engine by Charles Babbage
(1822-1834)

12
13
Intel Pentium Processor

14
15
16
 Computing Concepts

Processor
Chair Input Output Mr. Brown
Mr. Brown brought a chair
from the market
Office for his office
Market

DATA INFORMATION
Instructions

17
18
19
 Number Systems and Computer Arithmetic

Number System and their Conversions:

Positional Number System

(d p 1d p 2 d p 3 ... d1d 0 .d 1d 2 ... d n )b ,

Where, di {0,1,..., b 1}, i p 1, p 2,...,1, 0, 1,..., n;
n
Decimal value of the number is D di bi
i p 1

d p 1 is most Significant digit (MSD) d n Is least Significant digit (LSD)

Standard Number Systems

decimal (b=10), binary (b=2), octal (b=8), hexadecimal (b=16).20

 Number system Base/ radix Digits Examples
(b) {0, 1, , b-1}

Binary 2 0, 1 10012
Decimal 10 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
4910
Octal 8 0, 1, 2, 3, 4, 5, 6, 7
438
Hexadecimal 16 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
A, B, C, D, E, F 4C 216

Relationship between Standard Number Systems :

Binary Decimal Octal Hexadecimal 3-bit string
0 0 0 0 000
1 1 1 1 001
10 2 2 2 010
11 3 3 3 011 (Contd.)
100 4 4 4 100 21
Relationship between Standard Number Systems :

Binary Decimal Octal Hexadecimal 3-bit string

101 5 5 5 101
110 6 6 6 110
111 7 7 7 111
1000 8 10 8 ---
1001 9 11 9 ---
1010 10 12 A ---
1011 11 13 B ---
1100 12 14 C ---
1101 13 15 D ---
1110 14 16 E ---
1111 15 17 F ---
22
Conversions:

Hexadecimal Decimal:
1A2C16 1 163 10 162 2 161 12 160 670010
172810 6 162 C 161 0 160 6C 016

Octal Decimal:
427.58 4 82 2 81 7 80 5 81 279.62510
427.510 6 82 5 81 3 80 4 81 653.48

Binary Decimal:

1010102 1 25 0 24 1 23 0 22 1 21 0 20 4210
18210 1 27 0 26 1 25 1 24 0 23 1 22
1 21 0 20 101101102 23
 Fixed-point representation :

For an n-bit unsigned binary number, the general form is

q
d p 1 d p 2 d p 3 ... d 1 d 0 .d 1 d 2 ... d q
2
i
d i 2 , d i {0,1}, p q n .
i p 1 10
0
Case 1: If q = 0 and n = p d p 1 d p 2 d p 3 ... d 1 d 0
2
i
di 2 .
i p 1 10

q i
Case 1: If p = 1 and n = q+1 d 0 .d 1 d 2 ... d q
2
di 2 .
i0 10
Examples: X 1100000000000000.2 4914210
15
Take n = 16, x 1.1000000000000002 1.510 4914210 2

Therefore, x X 2 n 1. 2 n 1 Implicit scale factor

24
 Floating-point representation :

Basic Representation x f 10 E.

31 30 24 23 0

SIGN EXPONENT MANTISSA

7 bits 24 bits
1 bit

Examples: 0.00596 102 , 0.628 105 , 65.7452 E 1, 0.596 E 3

Normalized Representation

0.d1d 2 ..., d n 10 E , d i {0,1, 2,...,9}, d1 0.

Examples:
0.596 100 , 0.628 105 , 0.657452 E 1, 0.596 E 3
25
 Computer Arithmetic :

25 12 37,
12 25 13,
Integer Arithmetic
25 12 300,
25 12 2,
0.9996 103 0.9945 103 12 25 0.
0.0050 10 3 0.5000 105 ,
0.735 E 2 0.635 E 2
1.370 E 2 0.137 E 3,
0.200 E 2 0.400 E 2 Floating Point Arithmetic
0.080 E 4 0.800 E 3,
0.250 102 0.200 10 3
1.25 101 0.125 102.
26
 Errors in Arithmetic :
24 127
2 126 0 2 126 (1 224 ) * 2127
(1 2 )*2

Positive Positive
Negative Negative Negative underflow positive overflow
overflow numbers underflow numbers

Numbers in the IEEE standard format

Examples : 0.4632 E 99 0.5427 E 99 0.1005 E100 (a)

0.9627 E 99 0.9622 E 99 0.5000 E 102 (b)

If we assume that the exponent can have maximum two digits, then the result
(a) shows overflow
(b) Shows underflow
27
Approximations and Errors in Computing

SIGNIFICANT DIGITS :
22
3.1415926535897932384626...
7
1 2
0.333333333333... 0.285714285714...
3 7

3.141 four significant digits

1
0.333 three significant digits
3
2
0.285714 six significant digits
7
28
 Notion of Significant Digits :

All non-zero digits are significant.

All zeros occurring between non-zero digits are significant.
Trailing zeros following a decimal point are significant.
Zeros between the decimal point and preceding a non-zero digit are not significant.

When the decimal point is not written, trailing zeros are not considered.

Examples : Three significant digits: 3.50, 65.0, 0.230.

Three significant digits: 0.0123, 0.00123, 1.20.

significant digits ?? 7.56, 7.5600, 0.756, 756.0, 7.056, 70.56, 0756.

Accuracy and Precision: o The number 57.396 is accurate to five significant digits.
o The number has a precision of 0.001 or 103. 29
INHERENT ERRORS :

Data Errors Experimental measurements Limited accuracy

Conversion Errors Representation Errors Computer limitations

0.110 0.00011001
Example: 0.410 0.01100110
Sum 0.01111111
0.25 0.125 0.0625 0.03125
0.015625 0.0078125 0.000390625
0.49609375( 0.5)

Error is equal to 28 0.00390625.

30
NUMERICAL ERRORS : Chopping
(i) Roundoff Errors
Symmetric rounding

42.78 Abs. Err. 10 E d .

Example:
42.7893

42.79 Abs. Err. 0.5 10 E d .

(ii) Truncation Errors n

i
S ai x i
a x i

i 0 i 0

(0.2) 2 (0.2)3 (0.2) 4

e 0.2
1 0.2 1.22 Truncation. Err. 0.14 102.
2! 3!
4!
Truncate 31
OTHER ERRORS :

(i) Modelling Errors Missing Information

(ii) Blunders Caused due to human inperfection

(iii) Absolute Errors ea True value Approx. value

(iv) Relative Errors Abs. Err.

er
True value

(v) Percentage Errors er 100

32
 Taxonomy of errors
Total Error

Modelling Inherent Numerical Blunders

Errors Errors Errors

Missing Human
information imperfection

Data Conversion Roundoff Truncation

Errors Errors Errors Errors

Measuring Computing Numerical

method machine method 33
 Error Propagation, Conditioning & Stability

Error Propagation: Let xt xa ex and yt ya e y .

xt yt xa ex ya ey xt yt xa ex ya ey
xa ya ex ey . xa ya ex ey .

Total error ex y ex e y Total error ex y ex e y

Error bounds e x y ex ey

xt yt xa ex ya e y xa ya ya ex xa ey ex e y
e e
xa ya ya ex xa e y xa ya xa ya x y
xa ya
Total error e
xy

34
xt xa ex xa ya ya ex xa e y ex ey

yt ya e y ya2 ey2

xa ya ya ex xa ey
y 2

xa xa

ya ya
ex
xa


ey
ya
Total error e
x/ y
a

xa ex ey
ex / y
ya xa ya
Triangle inequality theorem
ex ey
exy xa ya
x ya
a
Errors introduced by the operations xa ya xa ya eo

xt yt xa ya ex ey eo

Total error ex y ex ey eo
35
Relative errors:
ex e y
er , x y
xa ya
Addition and Subtraction
xa ya
. er , x . er , y
xa ya xa ya

er , xy er , x er , y
Multiplication and division
er , x / y er , x er , y

Example:

Estimate the relative error in z = x-y when x = 0.1234 104 and y = 0.1232 104
as stored in a system with four-digit mantissa.

36
 1
We Know, er , x 103 0.05%
ex e y 2
er , z
x y 1
er , y 103 0.05%
2

Then,
ex 0.1234 104 0.5 103 0.617
e y 0.1232 104 0.5 103 0.616

Therefore,
ex y ex ey 1.233

1.233 104
er , z 0.6165 61.65%
0.1234 0.1232

37
 Process graph for u x y z u

x y z
er , u er , x y er , z r2
x y z x yz *

x y
er , x y er , x er , y r1 +
x y x y
z

x
y

In computing x y z x y z
38
 Error Calculation for a Polynomial Function f(x)

Let, f ( x) is differentiable and approximate value x0 of x is given.

According to mean-value theorem f ( x) f ( xa ) x xa f '

Error in f ( x) e f f ( x) f ( xa ) ex f '
ex max f '

e f ex f ' xa

Similarly, e f ex f x xa , ya e y f y xa , ya
39
Example:
Estimate the error in evaluating f ( x, y ) x 2 y 2 for x 3.00 and y 4.00.

Assume that, ex e y 0.005

f x ( x, y ) 2 x and f y ( x, y ) 2 y

Therefore, e f 2 x ex 2 y e y
(2 3.00 2 4.00) 5 103 0.07

40
 Error Analysis:

Original problem f ( x) y Actually Solved problem f ( x ) y

Absolute error y y f ( x) f ( x )

y f (x )
Exact

x
y
Calculated
x
Exact y f ( x)
x Results
Input Data

Forward Error: y ( y y) Backward Error x ( x x) 41

Conditioning and Stability Model

Sensitivity
of problem

Change in Numerical
Input data instability

Sensitivity of
numerical method

Induced error

Relative error in f ( x) er , f f x xf ' ( x)

Condition number .
Relative error in x er , x f ( x) x f ( x)
42
Example of ill-conditioned problem:

y m1 x C1 ,
Consider the system of equations where m1 and m2 are nearly equal
y m2 x C2 ,

C1 C2 C1 C2
Solving, x ; y m1 C1
m2 m1 m2 m1

Let, C1 7.00, C2 3.00, m1 2.00, m2 2.01.

7.00 3.00
x 400, y 2.00 400 7.00 807,
2.01 2.00

Let, C1 7.00, C2 3.00, m1 2.00, m2 2.005.

7.00 3.00
x 800, y 2.00 800 7.00 1607.
2.005 2.00 43
Example: Compute the condition number for f ( x ) ( x 1) .

Here, 1
f ( x) ( x 1) 1/ 2
'

xf ' ( x) x ( x 1) 1/ 2
Condition number
f ( x) 2 ( x 1)1/ 2
x

2( x 1)

Note: The function is numerically unstable for the values of x close to 1 .

44