Linear Systems – Iterative

methods

1. Jacobi Method
2. Gauss-Siedel Method

1
 Iterative Methods
Iterative methods can be expressed in the
general form: x(k) =F(x(k-1))
where s s.t. F(s)=s is called a Fixed Point

Hopefully: x(k)  s (solution of my problem)

 Will it converge? How rapidly?

2
 Iterative Methods
Stationary:
x(k+1) =Gx(k)+c
where G and c do not depend on iteration
count (k)

Non Stationary:
x(k+1) =x(k)+akp(k)
where computation involves information that
change at each iteration
3
 Iterative – Stationary
Jacobi
In the i-th equation solve for the value of xi while assuming the
other entries of x remain fixed:

bi   mij x j bi   mij x j
( k 1)
N

 mij x j  bi  xi  j i

j i
(k )
xi
j 1 mii mii

In matrix terms the method becomes: x (k )

 D 1
 L  U  x  k 1
 D 1
b

where D, -L and -U represent the diagonal, the strictly lower-

trg and strictly upper-trg parts of M
4
 Iterative – Stationary
Gauss-Seidel
Like Jacobi, but now assume that previously computed results
are used as soon as they are available:

bi   mij x j bi   mij x j   mij x j

(k ) ( k 1)
N

 mij x j  bi  xi  j i

j i j i
(k )
xi
j 1 mii mii

In matrix terms the method becomes: x ( k )   D  L  (Ux  k 1  b )

1

where D, -L and -U represent the diagonal, the strictly lower-

trg and strictly upper-trg parts of M
5
 Iterative – Stationary
Successive Overrelaxation (SOR)
Devised by extrapolation applied to Gauss-Seidel in the form of
weighted average:
bi   mij x j   mij x j
(k ) ( k 1)

( k 1) j i j i
 wxi  (1  w) xi 
(k ) (k ) (k )
xi xi
mii

In matrix terms the method becomes:

x ( k )   D  wL  ( wU  (1  w) D ) x  k 1  w( D  wL ) 1 b
1

where D, -L and -U represent the diagonal, the strictly lower-

trg and strictly upper-trg parts of M
w is chosen to increase convergence
6
 Jacobi iteration
a11x1  a12 x2    a1n xn  b1  x10 
a21x1  a22 x2    a2 n xn  b2  0
 x2 
x 
0
 
 0
an1 x1  an 2 x2    ann xn  bn  xn 

x11 
1
(b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi  bi 
aii 
 aij x   aij x 
k
j
k
j

1
j 1 j  i 1 
x12  (b2  a21 x10  a23 x30    a2 n xn0 )
a22
1
x1n  (bn  an1 x10  an 2 x20    ann1 xn01 )
ann
7
 Gauss-Seidel (GS) iteration
a11x1  a12 x2    a1n xn  b1  x10 
Use the latest a21x1  a22 x2    a2 n xn  b2  0
 x2 
update x 
0
 
 0
an1 x1  an 2 x2    ann xn  bn  xn 

x11 
1
(b1  a12 x20    a1n xn0 ) 1  i 1 n 
a11
k 1
xi  bi 
aii 
a ij x k 1
j   aij x  k
j
1 j 1 j  i 1 
x2 
1
(b2  a21 x1  a23 x3    a2 n xn )
1 0 0

a22
1
x1n  (bn  an1 x11  an 2 x12    ann1 x1n 1 )
ann
8
 Gauss-Seidel Method
An iterative method.

Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.

9
 Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.

Elimination methods such as Gaussian Elimination and LU

Decomposition are prone to prone to round-off error.

Also: If the physics of the problem are understood, a close initial

guess can be made, decreasing the number of iterations needed.

10
 Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
If: the diagonal elements are
a11x1  a12 x2  a13 x3  ...  a1n xn  b1 non-zero
a21x1  a22 x2  a23 x3  ...  a2n xn  b2 Rewrite each equation solving
. .
. . for the corresponding unknown
. .
ex:
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
First equation, solve for x1
Second equation, solve for x2

11
 Gauss-Seidel Method
Algorithm
Rewriting each equation
c1  a12 x2  a13 x3   a1n xn
x1  From Equation 1
a11

c2  a21 x1  a23 x3   a2 n xn
x2  From equation 2
a22
  
cn 1  an 1,1 x1  an 1, 2 x2   an 1,n  2 xn  2  an 1,n xn From equation n-1
xn 1 
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
xn  From equation n
ann

12
 Gauss-Seidel Method
Algorithm
General Form of each equation
n

a
n
c1   a1 j x j cn 1  n 1, j xj
j 1 j 1
j  n 1
x1 
j 1 xn 1 
a11 an 1,n 1
n
c n   a nj x j
n
c2   a2 j x j
j 1 j 1
j n
x2 
j 2
xn 
a 22 a nn

13
 Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci   aij x j
j 1
j i
xi  , i  1,2,, n.
aii

How or where can this equation be used?

14
 Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X] Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
 x1  most recent value of xi. Which
x  means to apply values calculated to
 2 the calculations remaining in the
current iteration.
  
 
 xn -1 
 xn 

15
 Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
xinew  xiold
a i  new
100
xi
So when has the answer been found?

The iterations are stopped when the absolute relative

approximate error is less than a prespecified tolerance for all
unknowns.

16
 Suppose that for conciseness we limit ourselves to a
33 set of equations.
j 1 j 1
b  a x  a x
x1j  1 12 2 13 3
(11.1a)
a11
b2  a21 x1j  a23 x3j 1
x 
2
j
(11.1b)
a22
b  a x j
 a x j
x3j  3 31 1 32 2
(11.1c)
a33
where j and j -1 are the present and previous iterations.
To start the solution process, initial guesses must be
made for the x’s. A simple approach is to assume that
they are all zero.

17
Convergence can be checked
using the criterion that for i,

xij  xij 1
 a ,i  j
100%   s
xi

18
 Graphical depiction of the difference between (a) the Gauss-Seidel and (b) the
Jacobi iterative methods for solving simultaneous linear algebraic equations.

19
 Jacobi Iterative Technique

Consider the following set of

equations.
10 x1  x2  2 x3 6
 x1  11x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3x2  x3  8 x4  15

20
 Convert the set Ax = b in the form of x =
Tx + c.
1 1 3
x1  x2  x3 
10 5 5
1 1 3 25
x2  x1  x3  x4 
11 11 11 11
1 1 1 11
x3   x1  x2  x4 
5 10 10 10
3 1 15
x4   x2  x3 
8 8 8

21
 Start with an initial approximation of:
 0, x2  0, x3  0 and x4  0.
( 0) ( 0) ( 0) ( 0)
x 1

1 (0) 1 (0) 3
  x3 
(1)
x1 x2
10 5 5
1 (0) 1 (0) 3 (0) 25
  x3  x4 
(1)
x2 x1
11 11 11 11
1 (0) 1 (0) 1 (0) 11
  x1   
(1)
x3 x2 x4
5 10 10 10
3 (0) 1 (0) 15
  x2  x3 
(1)
x4
8 8 8

22
 1 1 3
  (0) 
(1)
x1 (0)
10 5 5
1 1 3 25
  (0)  (0) 
(1)
x2 (0)
11 11 11 11
1 1 1 11
  (0)  (0)  (0) 
(1)
x3
5 10 10 10
3 1 15
  (0)  (0) 
(1)
x4
8 8 8

 0.6000, x 2  2.2727,
(1) (1)
x1

 1.1000 and x 4  1.8750

(1) (1)
x3
23
 1 (1) 1 (1) 3
  x3 
(2)
x1 x2
10 5 5
1 (1) 1 (1) 3 (1) 25
  x3  x4 
( 2)
x2 x1
11 11 11 11
1 (1) 1 (1) 1 (1) 11
  x1   
( 2)
x3 x2 x4
5 10 10 10
3 (1) 1 (1) 15
  x2  x3 
( 2)
x4
8 8 8

24
 1 ( k 1 ) 1 ( k 1 ) 3
x1   x3 
(k)
x2
10 5 5
1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
  x3  x4 
(k)
x2 x1
11 11 11 11
1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
  x1   
(k)
x3 x2 x4
5 10 10 10
3 ( k 1 ) 1 ( k 1 ) 15
  x2  x3 
(k)
x4
8 8 8

25
 Results of Jacobi Iteration:

k 0 1 2 3
(k ) 0.0000 0.6000 1.0473 0.9326
x1
(k ) 0.0000 2.2727 1.7159 2.0530
x2
(k ) 0.0000 -1.1000 -0.8052 -1.0493
x3
(k ) 0.0000 1.8750 0.8852 1.1309
x4

26
Gauss-Seidel Iterative Technique

Consider the following set of

equations.
10 x1  x2  2 x3 6
 x1  11x2  x3  3x4  25
2 x1  x2  10 x3  x4  11
3x2  x3  8 x4  15

27
 1 ( k 1 ) 1 ( k 1 ) 3
  x3 
(k)
x1 x2
10 5 5
1 ( k 1 ) 1 ( k 1 ) 3 ( k 1 ) 25
  x3  x4 
(k )
x2 x1
11 11 11 11
1 ( k 1 ) 1 ( k 1 ) 1 ( k 1 ) 11
  x1   
(k )
x3 x2 x4
5 10 10 10
3 ( k 1 ) 1 ( k 1 ) 15
  x2  x3 
(k )
x4
8 8 8

28
 1 ( k 1 ) 1 ( k 1 ) 3
x1   x3 
(k)
x2
10 5 5
1 (k) 1 ( k 1 ) 3 ( k 1 ) 25
  x3  x4 
(k)
x2 x1
11 11 11 11
1 (k) 1 (k) 1 ( k 1 ) 11
  x1   
(k)
x3 x2 x4
5 10 10 10
3 (k) 1 (k) 15
  x2  x3 
(k)
x4
8 8 8

29
 Results of Gauss-Seidel Iteration:
(Blue numbers are for Jacobi iterations.)
k 0 1 2 3
(k ) 0.0000 0.6000 1.0300 1.0065
x1
0.6000 1.0473 0.9326
(k ) 0.0000 2.3272 2.0370 2.0036
x2 2.2727 1.7159 2.0530
(k ) 0.0000 -0.9873 -1.0140 -1.0025
x3 -1.1000 -0.8052 -1.0493
(k ) 0.0000 0.8789 0.9844 0.9983
x4 1.8750 0.8852 1.1309

30
 The solution is: x1= 1, x2 = 2, x3 = -1, x4 = 1

It required 15 iterations for Jacobi

method and 7 iterations for Gauss-Seidel
method to arrive at the solution with a
tolerance of 0.00001.

While Jacobi would usually be the

slowest of the iterative methods, it
is well suited to illustrate an
algorithm that is well suited for
parallel processing!!!

31
 EXAMPLE Gauss-Seidel Method
Problem Statement. Use the Gauss-Seidel method to
obtain the solution for
3x1  0.1x2  0.2 x3  7.85
0.1x1  7 x2  0.3x3  19.3
0.3x1  0.2 x2  10 x3  71.4

Note that the solution is x

T
 3  2.5 7

Solution. First, solve each of the equations for its

unknown on the diagonal:

32
 7.85  0.1x2  0.2 x3
x1  (E11.1.1)
3
 19.3  0.1x1  0.3x3
x2  (E11.1.2)
7
71.4  0.3x1  0.2 x2
x3  (E11.1.3)
10
By assuming that x2 and x3 are zero
7.85  0.10  0.20
x1   2.616667
3
This value, along with the assumed value of x3 =0,
can be substituted into Eq.(E11.1.2) to calculate
 19.3  0.12.616667   0.30
x2   2.794524
7

33
 The first iteration is completed by substituting the
calculated values for x1 and x2 into Eq.(E11.1.3) to yield
71.4  0.32.616667   0.2 2.794524
x3   7.005610
10
For the second iteration, the same process is repeated to
compute
7.85  0.1 2.794524  0.27.005610
x1   2.990557
3
 19.3  0.12.990557   0.37.005610
x2   2.499625
7
71.4  0.32.990557   0.2 2.499625
x3   7.000291
10

34
 The method is, therefore, converging on the true
solution. Additional iterations could be applied to
improve the answers. Consequently, we can estimate
the error. For example , for x1

2.990557  2.616667
 a ,1  100%  12.5%
2.990557

For x2 and x3 , the error estimates are

 a , 2  11.8%
 a ,3  0.076%

Repeat to it again until the result is known to at

least the tolerance specified by s.

35
 Example: Unbalanced three phase load
Three-phase loads are common in AC systems. When the system
is balanced the analysis can be simplified to a single equivalent circuit
model. However, when it is unbalanced the only practical solution
involves the solution of simultaneous linear equations. In a model the
following equations need to be solved.
0.7460  0.4516 0.0100  0.0080 0.0100  0.0080  I ar   120 
0.4516 0.7460
 0.0080 0.0100 0.0080 0.0100   I ai   0.000 
0.0100  0.0080 0.7787  0.5205 0.0100  0.0080  I br   60.00
    
0.0080 0.0100 0.5205 0.7787 0.0080 0.0100   I bi    103.9 
0.0100  0.0080 0.0100  0.0080 0.8080  0.6040  I cr   60.00
    
0.0080 0.0100 0.0080 0.0100 0.6040 0.8080   ci   103.9 
I

Find the values of Iar , Iai , Ibr , Ibi , Icr , and Ici using the Gauss-Seidel
method.
36
 Example: Unbalanced three phase load
Rewrite each equation to solve for each of the unknowns
120.00   0.4516I ai  0.0100 I br   0.0080I bi  0.0100 I cr   0.0080I ci
I ar 
0.7460
0.000  0.4516 I ar  0.0080 I br  0.0100 I bi  0.0080 I cr  0.0100 I ci
I ai 
0.7460
 60.00  0.0100 I ar   0.0080I ai   0.5205I bi  0.0100 I cr   0.0080I ci
I br 
0.7787
 103.9  0.0080 I ar  0.0100 I ai  0.5205I br  0.0080 I cr  0.0100 I ci
I bi 
0.7787
 60.00  0.0100 I ar   0.0080I ai0.0100 I br   0.0080I bi   0.6040I ci
I cr 
0.8080

103.9  0.0080 I ar  0.0100 I ai  0.0080 I br  0.0100 I bi  0.6040 I cr

I ci 
0.8080
37
Example: Unbalanced three phase load
For iteration 1, start with an initial guess value

 I ar  20
 I  20
 ai   
Initial Guess:  I br  20
  
 I bi  20
 I cr  20
   
 I ci  20

38
 Example: Unbalanced three phase load
Substituting the guess values into the first equation
120   0.4516I ai  0.0100 I br   0.0080I bi  0.0100 I cr   0.0080I ci
I ar 
0.7460
 172.86

Substituting the new value of Iar and the remaining guess values into
the second equation

0.00  0.4516 I ar  0.0080 I br  0.0100 I bi  0.0080 I cr  0.0100 I ci

I ai 
0.7460
 105.61

39
 Example: Unbalanced three phase load
Substituting the new values Iar , Iai , and the remaining guess values into
the third equation
 60.00  0.0100 I ar   0.0080I ai   0.5205I bi  0.0100 I cr   0.0080I ci
I br 
0.7787
 67.039

Substituting the new values Iar , Iai , Ibr , and the remaining guess values into
the fourth equation
 103.9  0.0080 I ar  0.0100 I ai  0.5205I br  0.0080 I cr  0.0100 I ci
I bi 
0.7787
 89.499

40
 Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , Ibi , and the remaining guess
values into the fifth equation

 60.00  0.0100 I ar   0.0080I ai0.0100 I br   0.0080I bi   0.6040I ci

I cr 
0.8080
 62.548

Substituting the new values Iar , Iai , Ibr , Ibi , Icr , and the remaining guess
value into the sixth equation

103.9  0.0080 I ar  0.0100 I ai  0.0080 I br  0.0100 I bi  0.6040 I cr

I ci 
0.8080
 176.71

41
 Example: Unbalanced three phase load
At the end of the first iteration, the solution matrix is:

 I ar   172.86 
 I    105.61
 ai   
I br   67.039
  
I
 bi    89.499 
 I cr   62.548
   
 ci  
I 176 .71 

How accurate is the solution? Find the absolute relative

approximate error using:
xinew  xiold
a i  new
100
xi

42
 Example: Unbalanced three phase load
Calculating the absolute relative approximate errors
172.86  20  62.548  20
a 1  100  88.430% a 5  100  131.98%
172.86  62.548

 105.61  20 176.71  20
a 2  100  118.94% a 6  100  88.682%
 105.61 176.71

 67.039  20
a 3  100  129.83% The maximum error after
 67.039
the first iteration is:

 89.499  20 131.98%
a  100  122.35%
4
 89.499 Another iteration is needed!

43
 Example: Unbalanced three phase load
 I ar   172.86 
 I    105.61 
Starting with the values obtained in iteration #1  ai   
 I br   67.039
  
I
  bi  89.499 
 I cr    62.548
   
 I ci   176.71 

Substituting the values from Iteration 1 into the first equation

120.00   0.4516I ai  0.0100 I br   0.0080I bi  0.0100 I cr   0.0080I ci

I ar 
0.7460
 99.600

44
 Example: Unbalanced three phase load
Substituting the new value of Iar and the remaining values from
Iteration 1 into the second equation

0.00  0.4516 I ar  0.0080 I br  0.0100 I bi  0.0080 I cr  0.0100 I ci

I ai 
0.7460
 60.073

Substituting the new values Iar , Iai , and the remaining values from
Iteration 1 into the third equation

 60.00  0.0100 I ar   0.0080I ai   0.5205I bi  0.0100 I cr   0.0080I ci

I br 
0.7787
 136.15

45
 Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , and the remaining values from
Iteration 1 into the fourth equation
 103.9  0.0080 I ar  0.0100 I ai  0.5205I br  0.0080 I cr  0.0100 I ci
I bi 
0.7787
 44.299

Substituting the new values Iar , Iai , Ibr , Ibi , and the remaining values
From Iteration 1 into the fifth equation

 60.00  0.0100 I ar   0.0080I ai0.0100 I br   0.0080I bi   0.6040I ci

I cr 
0.8080
 57.259

46
 Example: Unbalanced three phase load
Substituting the new values Iar , Iai , Ibr , Ibi , Icr , and the remaining
value from Iteration 1 into the sixth equation

103.9  0.0080 I ar  0.0100 I ai  0.0080 I br  0.0100 I bi  0.6040 I cr

I ci 
0.8080
 87.441
 I ar   99.600 
 I    60.073
The solution matrix at the end of  ai   
the second iteration is:  I br    136.15 
  
I
 bi    44.299 
 I cr   57.259 
   
 I ci   87.441 

47
 Example: Unbalanced three phase load
Calculating the absolute relative approximate errors for
the second iteration
57.259  (62.548)
a 5  100  209.24%
99.600  172.86 57.259
a 1  100  73.552%
99.600
87.441  176.71
a 6  100  102.09%
 60.073  (105.61) 87.441
a 2  100  75.796%
 60.073
The maximum error after
a 3 
 136.35  (67.039)
100  50.762% the second iteration is:
 136.35
209.24%
 44.299  (89.499)
a 4  100  102.03% More iterations are needed!
 44.299

48
 Example: Unbalanced three phase load
Repeating more iterations, the following values are obtained
Iteration Iar Iai Ibr Ibi Icr Ici
1 172.86 −105.61 −67.039 −89.499 −62.548 176.71
2 99.600 −60.073 −136.15 −44.299 57.259 87.441
3 126.01 −76.015 −108.90 −62.667 −10.478 137.97
4 117.25 −70.707 −119.62 −55.432 27.658 109.45
5 119.87 −72.301 −115.62 −58.141 6.2513 125.49
6 119.28 −71.936 −116.98 −57.216 18.241 116.53

Iteration a 1 % a 2 % a 3 % a 4 % a 5 % a 6 %
1 88.430 118.94 129.83 122.35 131.98 88.682
2 73.552 75.796 50.762 102.03 209.24 102.09
3 20.960 20.972 25.027 29.311 646.45 36.623
4 7.4738 7.5067 8.9631 13.053 137.89 26.001
5 2.1840 2.2048 3.4633 4.6595 342.43 12.742
6 0.49408 0.50789 1.1629 1.6170 65.729 7.6884
49
 Example: Unbalanced three phase load
 I ar   119.28 
 I   71.936
After six iterations,  ai    The maximum error after
the solution matrix is  I br    116.98  the sixth iteration is:
  
I
 bi   57.216  65.729%
 I cr   18.241 
   
 I ci   116.53 

The absolute relative approximate error is still high, but allowing for
more iterations, the error quickly begins to converge to zero.
What could have been done differently to allow for a faster
convergence?

50
 Example: Unbalanced three phase load
Repeating more iterations, the following values are obtained

Iteration Iar Iai Ibr Ibi Icr Ici

32 119.33 −71.973 −116.66 −57.432 13.940 119.74
33 119.33 −71.973 −116.66 −57.432 13.940 119.74

Iteration a 1 % a 2 % a 3 % a 4 % a 5 % a 6 %

32 3.0666×10−7 3.0047×10−7 4.2389×10−7 5.7116×10−7 2.0941×10−5 1.8238×10−6

33 1.7062×10−7 1.6718×10−7 2.3601×10−7 3.1801×10−7 1.1647×10−5 1.0144×10−6

51
 Example: Unbalanced three phase load
 I ar   119.33 
 I   71.973
After 33 iterations, the solution matrix is  ai   
 I br    116.66 
  
I
 bi    57.432 
 I cr   13.940 
   
 I ci   119.74 

The maximum absolute relative approximate error is 1.1647×10−5%.

52
 Gauss-Seidel Method: Pitfall
Even though done correctly, the answer may not converge to the
correct answer.
This is a pitfall of the Gauss-Siedel method: not all systems of
equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.

Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:

n n
aii   aij for all ‘i’ and aii   aij for at least one ‘i’
j 1 j 1
j i j i
53
 Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at
least equal to the sum of the other coefficients in that row and at least
one row with a diagonal coefficient greater than the sum of the other
coefficients in that row.
Which coefficient matrix is diagonally dominant?

 2 5.81 34 124 34 56 

A   45 43 1  [B]   23 53 5 
123 16 1   96 34 129

Most physical systems do result in simultaneous linear equations that

have diagonally dominant coefficient matrices.

54
 Gauss-Seidel Method: Example 2
Given the system of equations The coefficient matrix is:
12 x1  3x2- 5x3  1 12 3  5
x1  5 x2  3x3  28 A   1 5 3 
3x1  7 x2  13x3  76  3 7 13 

With an initial guess of Will the solution converge using the

Gauss-Siedel method?
 x1  1
 x   0 
 2  
 x3  1

55
 Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
a11  12  12  a12  a13  3   5  8
12 3  5
A   1 5 3  a22  5  5  a21  a23  1  3  4
 3 7 13 
a33  13  13  a31  a32  3  7  10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

56
 Gauss-Seidel Method: Example 2
Rewriting each equation With an initial guess of
12 3  5  a1   1   x1  1
 1 5 3  a   28  x   0 
   2    2  
 3 7 13  a3  76  x3  1

1  3 x 2  5 x3 1  30  51
x1  x1   0.50000
12 12

28  x1  3 x3 28  0.5  31
x2  x2   4.9000
5 5

76  3 x1  7 x 2 76  30.50000  74.9000
x3  x3   3.0923
13 13

57
 Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000  1.0000
a 1  100  100.00%
0.50000

4.9000  0
a 2
 100  100.00%
4.9000

3.0923  1.0000
a 3
 100  67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%

58
 Gauss-Seidel Method: Example 2
After Iteration #1
 x1  0.5000
 x   4.9000
 2  
 x3  3.0923

Substituting the x values into the After Iteration #2

equations
 x1  0.14679
1  34.9000  53.0923  x    3.7153 
x1   0.14679
12  2  
 x3   3.8118 
28  0.14679  33.0923
x2   3.7153
5

76  30.14679  74.900 
x3   3.8118
13
59
 Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
0.14679  0.50000
a 1  100  240.61%
0.14679
3.7153  4.9000
a 2
 100  31.889%
3.7153
3.8118  3.0923
a 3
 100  18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?

60
 Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration a1 a 1 % a2 a 2 % a3 a 3 %

1 0.50000 100.00 4.9000 100.00 3.0923 67.662

2 0.14679 240.61 3.7153 31.889 3.8118 18.876
3 0.74275 80.236 3.1644 17.408 3.9708 4.0042
4 0.94675 21.546 3.0281 4.4996 3.9971 0.65772
5 0.99177 4.5391 3.0034 0.82499 4.0001 0.074383
6 0.99919 0.74307 3.0001 0.10856 4.0001 0.00101

 x1  0.99919  x1  1
The solution obtained  x    3.0001  is close to the exact solution of  x    3 .
 2    2  
 x3   4.0001   x3  4

61
 Gauss-Seidel Method: Example 3
Given the system of equations

3x1  7 x2  13x3  76 Rewriting the equations

x1  5x2  3x3  28 76  7 x2  13x3

x1 
12 x1  3x2  5 x3  1 3
28  x1  3 x3
With an initial guess of x2 
5
 x1  1
 x   0  1  12 x1  3 x 2
 2   x3 
 x3  1 5

62
 Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained

Iteration a1 a 1 % A2 a 2 % a3 a 3 %
1 21.000 95.238 0.80000 100.00 50.680 98.027
2 −196.15 110.71 14.421 94.453 −462.30 110.96
3 −1995.0 109.83 −116.02 112.43 4718.1 109.80
4 −20149 109.90 1204.6 109.63 −47636 109.90
5 2.0364×105 109.89 −12140 109.92 4.8144×105 109.89
6 −2.0579×105 109.89 1.2272×105 109.89 −4.8653×106 109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

63
 Gauss-Seidel Method
The Gauss-Seidel Method can still be used
 3 7 13 
The coefficient matrix is not
diagonally dominant
A   1 5 3 
12 3  5
But this is the same set of 12 3  5
equations used in example #2,
which did converge.
A   1 5 3 
 3 7 13 

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.

64
 Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations

x1  x2  x3  3
2 x1  3x2  4 x3  9
x1  7 x2  x3  9

Which equation(s) prevents this set of equation from having a

diagonally dominant coefficient matrix?

65