ITERATIVE METHODS FOR

SYSTEM OF EQUATIONS

SUBMITTED BY:

SONAL GUPTA
ROLL NO. 16114269
PHD IME
TYPES

• Linear System of Equations

• Jacobi Iteration
• Gauss Seidel
• Non Linear System of Equations
• Newton Raphson Method
NEWTON-RAPHSON METHOD
INTRODUCTION

In numerical analysis, Newton's method (also known as the Newton–

Raphson method), named after Isaac Newton and Joseph Raphson, is
a method for finding successively better approximations to the roots (or
zeroes) of a real-valued function. It is one example of a root-finding
algorithm.
 f(x)

f(xi)
x f x 
i, i

f(xi )
xi 1 = xi -
f (xi )
f(xi-1)


xi+2 xi+1 xi X

Figure 1 Geometrical illustration of the Newton-Raphson method

f(x)

f(xi) B
AB
tan(  
AC

f ( xi )
f ' ( xi ) 
xi  xi 1
C  A X
xi+1 xi
f ( xi )
xi 1  xi 
f ( xi )

Figure 2 Derivation of the Newton-Raphson method

ALGORITHM FOR NEWTON-RAPHSON METHOD
STEP 1

Evaluate f’(x) symbolically

STEP 2
Use an initial guess of the root, , to estimate the new value of the root,
xi
xi 1 , as
f  xi 
xi 1 = xi -
f  xi 
STEP 3

Find the absolute relative approximate error a as

xi 1- xi
a =  100
xi 1
STEP 4

• Compare the absolute relative approximate error with the pre-

specified relative error tolerance s
Go to Step 2 using new
YES estimate of the root

Is a s ?
NO Stop the algorithm

• Also, check if the number of iterations has exceeded the maximum

number of iterations allowed. If so, one needs to terminate the
algorithm and notify the user.
EXAMPLE 1
• You are working for ‘DOWN THE TOILET COMPANY’ that makes
floats for ABC commodes. The floating ball has a specific gravity of
0.6 and has a radius of 5.5 cm. You are asked to find the depth to
which the ball is submerged when floating in water.
• The equation that gives the depth x in meters to which the ball is
submerged under water is given by

f x   x3-0.165x 2+3.993 10- 4

• Solve for f ' x 
f x   x3-0.165x 2+3.993 10- 4
f ' x   3x 2-0.33x
• Let us assume the initial guess of the root of f x  0 is x0  0.05m
This is a reasonable guess (discuss why x  0 and x  0.11m are not
good choices) as the extreme values of the depth x would be 0 and
the diameter (0.11 m) of the ball.
ITERATION 1
THE ESTIMATE OF THE ROOT IS

f  x0 
x1  x0 
f '  x0 

 0.05 
0.05  0.1650.05  3.993  10  4
3 2

30.05  0.330.05
2

1.118  10  4
 0.05 
 9  10 3
 0.05   0.01242 
 0.06242
• The absolute relative approximate error a at the end of Iteration 1 is

x1  x0
a   100
x1
0.06242  0.05
  100
0.06242
 19.90%
• The number of significant digits at least correct is 0, as you need an
absolute relative approximate error of 5% or less for at least one
significant digits to be correct in your result.
ITERATION 2
THE ESTIMATE OF THE ROOT IS

f  x1 
x2  x1 
f '  x1 

 0.06242 
0.06242   0.1650.06242   3.993  10  4
3 2

30.06242   0.330.06242 
2

 3.97781 10 7
 0.06242 
 8.90973  10 3

 0.06242  4.4646  10 5
 0.06238
• The absolute relative approximate error a at the end of Iteration 2 is

x2  x1
a  100
x2
0.06238  0.06242
 100
0.06238
 0.0716%

• Hence, the number of significant digits at least correct in the answer is

2 because a <=0.5% .
ITERATION 3
THE ESTIMATE OF THE ROOT IS

f  x2 
x3  x2 
f '  x2 

 0.06238 
0.06238  0.1650.06238  3.993  10  4
3 2

30.06238  0.330.06238
2

4.44  10 11
 0.06238 
 8.91171 10 3
 
 0.06238   4.9822  10 9
 0.06238
• The absolute relative approximate error a at the end of Iteration 3 is

x2  x1
a  100
x2
0.06238  0.06238
  100
0.06238
 0%

• The number of significant digits at least correct is 4, as only 4

significant digits are carried through all the calculations.
• So , X3 = 0.06238 will be the roots of the equation.
GAUSS-SEIDEL METHOD
INTRODUCTION

• In numerical linear algebra, the Gauss–Seidel method, also known

as the Liebmann method or the method of successive
displacement, is an iterative method most commonly used to solve
a linear system of equations or simultaneous linear equations and to
find the roots of the equation or unknowns in equation. Though it can
be applied to any matrix with non-zero elements on the diagonals,
convergence is only guaranteed if the matrix is either diagonally
dominant, or symmetric and positive definite.
• The Gauss-Seidel Method allows the user to control round-off error.
ALGORITHM

A set of n equations and n unknowns:

a11x1 + a12x2 +a13x3 +………+a1nxn = C1
If: the diagonal elements are non-
a21x1 + a22x2 +a23x3 +………+a2nxn = C2 zero

a31x1 + a32x2 +a33x3 +………+a3nxn = C3 Rewrite each equation solving for

the corresponding unknown
. .
ex: First equation, solve for x1
. . Second equation, solve for x2

. .

an1x1 + an2x2 +an3x3 +………+annxn = Cn

ALGORITHM
Rewriting each equation
c  a x  a13 x3   a1n xn
x1  1 12 2 From Equation 1
a11
c2  a21 x1  a23 x3   a2 n xn
x2  From equation 2
a22
  
cn 1  an 1,1 x1  an 1, 2 x2   an 1,n  2 xn  2  an 1,n xn
xn 1  From equation n-1
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
xn  From equation n
ann
ALGORITHM
STEP 1
General Form for any row ‘i’
n
ci   aij x j
j 1
j i
xi  , i  1,2,, n.
aii
STEP 2
Assume an initial guess of roots for [X]

 x1  Use rewritten equations to solve for each

x  value of xi.
 2 Important: Remember to use the most
   recent value of xi. Which means to apply
  values calculated to the calculations
 xn -1  remaining in the current iteration.
 xn 
STEP 3

Calculate the Absolute Relative Approximate Error

xnew
x old
a i  i
new
i
100
x i

We will check for iterations.The iterations are stopped when the absolute relative
approximate error of max of Absolute relative approximate error is less than a pre
specified tolerance for all unknowns.
 EXAMPLE

Given the system of equations: The coefficient matrix is:

12x1 +3x2 -5x3 =1 12 3  5
x1 +5x2 + 3x3 = 28 A   1 5 3 
3x1 +7x2 + 13x3 = 76  3 7 13 

With an initial guess of:

 x1  1
[ X ] =  x 2   0 
   
 x3  1
Checking if the coefficient matrix is diagonally dominant:

a11  12  12  a12  a13  3   5  8

12 3  5
A   1 5 3 
 3 7 13 
a22  5  5  a21  a23  1  3  4

a33  13  13  a31  a32  3  7  10

The inequalities are all true and at least one row is strictly greater .
Therefore: The solution should converge using the Gauss-Siedel Method
STEP 1
Rewriting each equation With an initial guess of
 x1  1
 x   0 
 2  
 x3  1

1  3 x 2  5 x3 1  30  51
x1  x1   0.50000
12 12

28  x1  3 x3 28  0.5  31
x2  x2   4.9000
5 5

76  30.50000  74.9000
76  3 x1  7 x 2 x3   3.0923
x3  13
13
STEP 2
The absolute relative approximate error
0.50000  1.0000
a 1  100  100.00%
0.50000

4.9000  0
a 2
 100  100.00%
4.9000

3.0923  1.0000
a 3
 100  67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%
STEP 3
After Iteration #1
 x1  0.5000
 x   4.9000
 2  
 x3  3.0923

Substituting the x values into the equations

After Iteration #2
1  34.9000  53.0923
x1   0.14679
12  x1  0.14679
 x    3.7153 
28  0.14679  33.0923  2  
x2   3.7153  x3   3.8118 
5
76  30.14679  74.900 
x3   3.8118
13
Iteration #2 absolute relative approximate error
0.14679  0.50000
a 1  100  240.61%
0.14679

3.7153  4.9000
a 2
 100  31.889%
3.7153

3.8118  3.0923
a 3  100  18.874%
3.8118

The maximum absolute relative error after the first iteration is 240.61%
ITERATIONS #3,4 AND SO ON
Repeating more iterations, the following values are obtained

Iteration x1 a 1 % x2 x3
a 2 % a 3 %

1 0.50000 100.00 4.9000 100.00 3.0923 67.662

2 0.14679 240.61 3.7153 31.889 3.8118 18.876

3 0.74275 80.236 3.1644 17.408 3.9708 4.0042

4 0.94675 21.546 3.0281 4.4996 3.9971 0.65772

5 0.99177 4.5391 3.0034 0.82499 4.0001 0.074383

6 0.99919 0.74307 3.0001 0.10856 4.0001 0.00101

SOLUTION

The solution obtained  x1  0.99919 is close to the exact solution of  x1  1

 x    3.0001   x    3
 2    2  
 x3   4.0001   x3  4

So , X1 = 1 , X2 = 3 , X3 = 4 are the roots.

REFERENCES

• https://en.wikipedia.org/wiki/Newton%27s_method
• http://nm.mathforcollege.com/topics/newton_raphson.html
• https://www.sciencedirect.com/science/article/pii/0022247X66901156
• https://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method
• http://mathforcollege.com/nm/videos/youtube/04sle/gauss_seidel/gauss_se
idel_04sle_theory1.html
THANKS!