Report for Ex1; Ex2-a-b-c

Exercise 1
This Matlab result is provided for proving that the equation from positive definiteness of
matrix 1 is always bigger than zero for all-non-zero entries a, b & c. Below, reader can see the
graph generated for evaluating the following expression:
2 (2 + 2 + 2 ) ( + )

Fig1. 3D graphical evaluation of the above expression, proving that it is always bigger than zero for
otherwise all-non-zero entries for a, b & c.
Exercise 2 Part a
In this part, each iterative method is tested for both matrices to observe their behaviour.
Below, reader can find tables for resulting data.
Computational
Time

Jacobi Method 0.0074 0.0081

Gau-Seidel
SOR

0.0079 0.0092
0.011

0.0065

Table 1. Computational Time [s]

Total Iteration

Jacobi Method

100

Gau-Seidel

32

100

SOR

100

100

Table 2. Total number of iteration

1

As you can see, only Jacobi method for matrix A2, and Gau-Seidel for matrix A1
converged enough with respect to our tolerance. Other methods unfortunately failed to converge
for given maximum number of iteration. As we were expecting from the first exercise, all the
methods had some potential danger to diverge and 4 out of 6 methods really did. Reason for
this will become clearer in the 3 part of this exercise.
The computational elapsed times are a little bit misleading. Jacobi method is expected to
be more time consuming since it offers a slower convergence rate than Gau-Seidel method
and etc. However, since in our implementation we avoided taking inverses, this resulted more
operational effort for Gau-Seidel and SOR. Although, again we expect these two methods to
be faster in convergence, since the convergence criteria are not met for all, it is misleading to
compare these results (because some methods diverge). Additionally, matrix sizes are
practically quite small for these iterative methods. So the computation time is already too small
to compare and this creates another factor to make it harder to comment on the elapsed times.
Nonetheless, still the elapsed time for Gau-Seidel in 1st matrix (with 32 iterations), is still
slightly shorter than the elapsed time for Jacobi Method in 2nd matrix (4 iterations).
When we look at the total number of iterations, we can easily say which method
converged and which did not. Again a misleading result is, Jacobi method took only 4 steps to
reach the solution. On the other hand, Gau-Seidel took 32 steps to solve the other (but very
similar) system. This is due to the fact that each method has different convergence characteristic
for each matrix. One shouldnt directly assume that Jacobi method will be slower than e.g.
Gau-Seidel method. These comparisons highly depend on the application, briefly on the
matrix. So again, investigating spectral radii will give us crucial insight about why methods
behaved this way.
Finally below, reader can find the computed solutions for each method and matrix:
Jacobi Method

Gau-Seidel

SOR

4.0481 104
( 6.229 103 )
3.7368 104

0.2777
( 1.6111 )
0.1666

5.888 1033
(4.406 1033 )
4.164 1033

8
( 20 )
25

3.3301 1067
(1.3794 1067 )
9.419 1067

2.4153 10133
(2.8112 10133 )
1.8339 10133

Table 3. Computed final solutions for each method and matrix

Part b
In this part, exact solutions of the two systems were found by direct inversion of the
matrices. Below reader can find the solutions:
0.2777
1 = ( 1.6111 )
0.1666

&

8
2 = ( 20 )
25

As you can see, converged methods were able to find the exact solutions.
Part c
In this part the spectral radius (convergence speed), prior and posterior estimates for both
matrices under each method are computed.
Spectral Radii

Jacobi Method

1.118

1.0809 105

Gau-Seidel

0.5

4.8284

SOR

2.2179

22.5587

Table 4. Spectral Radii for each method corresponding to both matrices

As we can see here, the reason for why some methods converged and some did not,
became very clear. As we know, ultimate requirement for convergence is having spectral radius
for the given method and matrix to be less than one. Only Gau-Seidel for matrix 1 and Jacobi
for matrix 2 satisfy this condition. And they also indicate why Jacobi took very less number of
iterations: because the spectral radius is very small. Values in the table also explains why final
values of each method are very off compared to exact solution. For example, spectral radius for
SOR in matrix 2 is quite high, and that explains why we get a huge divergence in the same
number of maximum iterations compared to other methods.
We can think of the value of the spectral radius as the derivative/slope of a function. The
value compared to 1 tells us how our method is going to behave. Bigger the value than 1, faster
we will diverge. Smaller the value than 1 and close to zero, faster we will converge. Remember
that spectral radius equal to 0 ensures at least quadratic convergence, much faster than linear,
which we had in all of the methods we experimented here.
As for the prior and posterior estimates, we first need to compute the so called M, the
iteration matrix of the method. After finding matrix M, we need to find = and
3

establish our estimate inequalities. Also satisfying () < 1 & 1 will make our
inequalities true.
According to our Matlab computations, below are the values for q:
Infinity norm of
iteration matrix M

Jacobi Method

Gau-Seidel

14

SOR

3.59

61.39

Table 5. Infinity norm of each M matrix

The spectral radius values for iteration matrices are exactly the same with Table 4.
According to these results, only for Gau-Seidel method for first matrix, the prior and
posterior estimates seems to be true. However when we take the 1- norm of the M matrix for
Gau-Seidel in matrix 1, we find the value 1.5, which is incidentally bigger than one. So
basically for each method applied on both matrices, the prior and posterior estimates are not
true. Nevertheless we can represent them, as shown in the following:
Prior

Posterior:

() 2 (1) (0)
() 2 () (1)
Inequalities 1. Jacobi method Matrix 1

Prior

Posterior:

()

4
3

(1) (0)

() 3 () (1)
Inequalities 2. Jacobi method Matrix 2

()

Posterior:

()

Prior

(1) (0)
() (1)

Inequalities 3. Gau-Seidel Matrix 1

4

Prior

Posterior:

()

14
13

(1) (0)

14

() 13 () (1)
Inequalities 4. Gau-Seidel Matrix 2

Prior

Posterior:

()

3.59
2.59

(1) (0)

3.59

() 2.59 () (1)
Inequalities 5. SOR Matrix 1

Prior

Posterior:

()

61.39
60.39

(1) (0)

61.39

() 60.39 () (1)
Inequalities 6. SOR Matrix 2

Since we do not have strictly diagonally dominant matrices, above results are confusing.
We have a rather special example of matrices that are expected to cause troubles in iteration,
but somehow manage to provide solutions with some methods (Jacobi for second, Gau-Seidel
for first matrix).