Dr.

Satish Shukla 1 of 20

Engg. Math. I (Unit-IV)

Syllabus (Unit-IV): Numerical Analysis : Numerical Methods for Solving System

of Linear Equations: Norms; Condition Numbers, Forward Gaussian Elimination
and Backward Substitution; Gauss-Jordan Elimination; FGE with Partial Piv-
oting and Row Scaling; LU Decomposition; Iterative Methods: Jacobi, Gauss-
Seidel; Power method and QR method for Eigen Value and Eigen vector.

Norms and Condition Numbers of a Matrix

In this discussion, we consider every column matrix as a vector. For example, if v =

[v1 , v2 , v3 ]0 is a column matrix, then it can be represented by a three dimensional vector
~v = iv1 + jv2 + kv3 , or by a more simpler notation (v1 , v2 , v3 ). Similar to the magnitude
of a vector, we define the norm of a column matrix.
`p -norm of a vector. Let ~v = (v1 , v2 , . . . , vn ) be an n-dimensional vector, then we
denote the p-norm or `p -norm of the vector by k~v kp and:

k~v kp = (|v1 |p + |v2 |p + + |vn |p )1/p

where p is any number such that 1 p < . A vector ~v is called a unit vector (or
unit) if k~v k = 1.
Norm of a Matrix. Suppose, A be a matrix. Then we define the p-norm of matrix A
by:
kAkp = max kAU kp
(U is unit)

where max kAU kp denotes the maximum value of kAU kp , where maximum runs
(U is unit)
over U such that U is a unit.
The norm kAk1 , kAk2 and kAk are the three usual norms of a matrix (used most
frequently), where A = [aij ]nm and kAk is equal to maximum row-wise sum of
matrix, i.e.,
( m m m
)
X X X
kAk = max |a1j |, |a2j |, . . . , |anj | .
j=1 j=1 j=1

Although, the norm of a matrix can be defined in many other ways.

 
1 1
Example 1. Find the usual three norms of A = .
1 1

Sol. (i) Suppose U = [u1 u2 ]T be a unit vector, i.e.,

kU k1 = |u1 | + |u2 | = 1.
Dr. Satish Shukla 2 of 20

Then:
   
1 1 u1
kAk1 = max kAU k1 = max
(U is unit) (U is unit) 1 1 u2
1
 
u1 + u2
= max
(U is unit) u2 + u2 1

= max |u1 + u2 | + |u1 + u2 |

(|u1 |+|u2 |=1)
= 2.

(ii) Suppose U = [u1 u2 ]T be a unit vector, i.e.,

1/2
kU k2 = u21 + u22 = 1 or u21 + u22 = 1

Then:
   
1 1 u1
kAk2 = max kAU k2 = max
(U is unit) (U is unit) 1 1 u2
2
 
u1 + u2
= max
(U is unit) u2 + u2 2
1/2
(|u1 + u2 |)2 + (|u1 + u2 |)2

= max
(u21 +u22 =1)

1/2
2 u21 + u22 + 2u1 u2

= max
(u21 +u22 =1)
= 41/2 = 2.
1
since the maximum value of u1 u2 under the condition u21 + u22 = 1 is .
2
(iii)
( 2 2
)
X X
kAk = max |a1j |, |a2j | = max {1 + 1, 1 + 1}
j=1 j=1
= 2.

Note. (i) Matrix norm for `1 -norm is maximum absolute column sum, i.e.,
( n n n
)
X X X
kAk1 = max |ai1 |, |ai2 |, . . . , |aim | .
i=1 i=1 i=1

Condition Number of a matrix. We know that every matrix represents a trans-

formation. The condition number of a matrix is the measure of the sensitivity of
corresponding transformation. For example, if a unit sphere is transformed under a
matrix A, then the larger the condition number, the more distorted (relatively long and
thin) the unit sphere becomes when transformed by the matrix. The condition number
a measure of how close a matrix is to being singular: a matrix with large condition
number is nearly singular, whereas a matrix with condition number close to 1 is far
Dr. Satish Shukla 3 of 20

from being singular. Mathematically, the condition number of a nonsingular matrix A

with respect to the `p -norm is denoted by condp (A) or p (A) and:

condp (A) = p (A) = kAkp kA1 kp .

In two dimensions, for example, the unit circle in the `2 -norm becomes and increasingly
cigar shaped ellipse as the condition number increases.
 
1 1
Example 2. Find the condition number 1 of the matrix A = with
1 1
respect to the `1 -norm.
 
1 1 1 1 1
Sol. Here A = adjA = . Then:
|A| 0 1 1
( n n
)
X X
kAk1 = max |ai1 |, |ai2 | = max {|1| + |1|, |1| + |1|} = 2,
i=1 i=1

1
kA1 k1 = max {|1| + | 1|, | 1| + |1|} = .
0
Therefore,
(A) = kAk1 kA1 k1 = 2 = .
Note that, the matrix A is singular.

Solution of linear simultaneous equation (Direct Methods).

Gauss Elimination Method: In this method, the unknowns are eliminated successively
and the system is reduced to an upper triangular system from which the unknown are
found by back substitution.
Consider the following system of equations:

a11 x + a12 y + a13 z = b1 (1)

a21 x + a22 y + a23 z = b2 (2)
a31 x + a32 y + a33 z = b3 . (3)

To solve this system, we follow the following steps:

Step I. First we eliminate the variable x from equation (2) and (3) with the help of
equation (1). For this, we choose the coefficient of x in equation (1), it is a11 . Now,
a21 a31
we apply the operation (2) (1) and (3) (1) and obtain:
a11 a11
a11 x + a12 y + a13 z = b1
a022 y + a023 z = b02 (4)
a032 y + a033 z = b03 . (5)

Here, the equation (1) is called the pivotal equation and the coefficient a11 is called the
first pivot.
Dr. Satish Shukla 4 of 20

Step II. Now leave the first equation unchanged and eliminate the variable y from
equation (5) with the help of equation (4). For this, we choose the coefficient of y in
a0
equation (4), it is a011 . Now, we apply the operation (5) 032 (4) and obtain:
a22
a11 x + a12 y + a13 z = b1
a022 y + a023 z = b02
a0033 z = b003 . (6)
Here, the equation (4) is called the pivotal equation and the coefficient a022 is called the
new (second) pivot.
Step III. Now the values of variables can be obtained by back substitution.
Partial and complete pivoting. When applying the elimination method, we eliminate
the variables one-by-one from the equations. Suppose, we have to solve a system of
equations of three variables. namely, x, y and z. Its a matter of practice, we follow
the alphabetic order for elimination, i.e., first we eliminate x from second and third
equations, and then y from the third equation. There is always a possibility of error
due to roundoff or truncation. To minimize it we use the process of pivoting. In the
first step, the numerically largest coefficient of x is chosen from all the equations and
brought as the first pivot by interchanging the first equation with the equation having
largest coefficient of x. In the second step, the numerically largest coefficient of y is
chosen from all the remaining equations (i.e., from all the equation except first) and
brought as the second pivot by interchanging the second equation with the equation
having largest coefficient of y. This process is continued till we arrive at the equation
with the single variable. This procedure is called the partial pivoting.
If we are not keen about the elimination of x, y, z in the alphabetic order, then we
choose at each stage the numerically largest coefficient of the entire system of equations,
and perform the whole process, then such procedure is called the complete pivoting.
Example 3. Find numerical solution of the system of equations given below by
forward Gauss elimination method (or Gauss elimination method) with partial
pivoting:
x+y+z = 9
2x 3y + 4z = 13
3x + 4y + 5z = 40.

Sol. The numerically largest coefficient of x is 3 in third equation, therefore 3 is the

first pivot and we rewrite the system as follows:
3x + 4y + 5z = 40 (7)
2x 3y + 4z = 13
x + y + z = 1.
1
Applying the operation (7):
3
x + 1.3333y + 1.6667z = 13.3333 (8)
2x 3y + 4z = 13 (9)
x + y + z = 1. (10)
Dr. Satish Shukla 5 of 20

To eliminate x from (9) and (9), applying (9) 2 (8) and (10) (8) we obtain:

x + 1.3333y + 1.6667z = 13.3333

5.6667y + 0.6666z = 13.6666 (11)
0.3333y 0.6667z = 4.3333. (12)

The numerically largest coefficient of y in the remaining, i.e., in equation (11) and (12) is
1
5.6667 (in equation (11)), therefore 5.6667 is the second pivot. Applying (11):
5.6667
x + 1.3333y + 1.6667z = 13.3333
y + 0.1176z = 2.4117 (13)
0.3333y 0.6667z = 4.3333. (14)

To eliminate y from (14), applying (14) 0.3333 (13) we obtain:

x + 1.3333y + 1.6667z = 13.3333 (15)

y + 0.1176z = 2.4117 (16)
0.7059z = 3.5294. (17)

Using back substitution it follows from the above equations that:

z = 5, y = 3, x = 1.

Gauss-Jordan Method. This method is a minor modification of the Gauss elimination

method. In this method, the given system is transformed into a diagonal system,
instead, an upper triangular system (as we do in Gauss elimination method). Therefore,
the back substitution is not required.

Example 4. Find numerical solution of the system of equations given below by

forward Gauss-Jordan method.
10x 7y + 3z + 5u = 6
6x + 8y z 4u = 5
3x + y + 4z + 11u = 2
5x 9y 2z + 4u = 7.

Sol. Rewrite the system in matrix form we get:

10 7 3 5 6
6 8 1 4 5

3 1 4 11 2
5 9 2 4 7.
1
Applying R1 R1 (divide first row by 10) we obtain:
10

1 0.7 0.3 0.5 0.6
6 8 1 4 5

3 1 4 11 2
5 9 2 4 7.
Dr. Satish Shukla 6 of 20

Applying R2 R2 + 6R1 , R3 R3 3R1 and R4 R4 5R1 :

1 0.7 0.3 0.5 0.6
0 3.8
0.8 1 8.6

0 3.1 3.1 9.5 0.2
0 5.5 3.5 1.5 4.
1
Applying R2 :
3.8
1 0.7 0.3 0.5 0.6
0
1 0.2152 0.2631 2.2631

0 3.1 3.1 9.5 0.2
0 5.5 3.5 1.5 4.
Applying R1 R1 + 0.7R2 R3 R3 3.1R2 and R4 R4 + 5.5R2 :

1 0 0.4473 0, 3157 2.1842
0
1 0.2152 0.2631 2.2631
0 0 2.4473 10.3157 6.8157
0 0 2.3421 0.0526 16.4473.
1
Applying R3 :
2.4473

1 0 0.4473 0.3157 2.1842
0
1 0.2152 0.2631 2.2631
0 0 1 4.2150 2.7849
0 0 2.3421 0.0526 16.4473.

Applying R1 R1 0.4473R3 , R2 R2 0.2152R3 and R4 R4 + 2.3421R3 :

1 0 0 1.5698 3.4301
0 1
0 1.1505 2.7526
0 0 1 4.2150 2.7849
0 0 0 9.9247 9.9247.
1
Applying R4 :
9.9247
1 0 0 1.5698 3.4301
0
1 0 1.1505 2.8494

0 0 1 4.2150 2.7849
0 0 0 1 1.
Applying R1 R1 + 1.5698R4 , R2 R2 + 1.1505R4 , and R3 R3 4.2150R4 :

1 0 0 0 4.9999
0 1 0 0 3.9999

0 0 1 0 6.9999
0 0 0 1 1.

Therefore,
x = 5, y = 4, z = 7, u = 1.
Dr. Satish Shukla 7 of 20

Crouts Triangularisation Method or LU Decomposition Method or Factorization

Method. Suppose we have to solve the following system of equations:

a11 x + a12 y + a13 z = b1

a21 x + a22 y + a23 z = b2
a31 x + a32 y + a33 z = b3

Then the above system can be written as:

AX = B (18)

a11 a12 a13 x b1
where A = a21 a23 a23 , X = y and B = b2 . Then, we decompose the
a31 a32 a33 z b3
matrix A as a product of a lower triangular matrix L and an upper triangular matrix
U , i.e., A = LU or

a11 a12 a13 1 0 0 u11 u12 u13
a21 a23 a23 = `21 1 0 0 u23 u23 .
a31 a32 a33 `31 `32 1 0 0 u33

By comparison we can find the value of `s and us. Then (18) becomes: LU X = B
or LV = B (writing U X = V ). Now these equations can be solved easily for x, y and z.

Example 5. Solve by LU decomposition method:

3x + 2y + 7z = 4
2x + 3y + z = 5
3x + 4y + z = 7.

Sol. Then the above system can be written as:

AX = B. (19)

3 2 7 x 4
Here A = 2 3 1 , X = y and B = 5 . Let LU = A, i.e.,
3 4 1 z 7

3 2 7 1 0 0 u11 u12 u13
2 3 1 = `21 1 0 0 u22 u23
3 4 1 `31 `32 1 0 0 u33

3 2 7 u11 u12 u13
= 2 3 1 = `21 u11 `21 u12 + u22 `21 u13 + u23 .
3 4 1 `31 u11 `31 u12 + `32 u22 + `31 u13 + `32 u23 + u33

On comparing we obtain:
Dr. Satish Shukla 8 of 20

(i) u11 = 3, u12 = 2, u13 = 7.

(ii) `21 u11 = 2, `21 u12 + u22 = 3, `21 u13 + u23 = 1.
(iii) `31 u11 = 3, `31 u12 + `32 u22 = 4, `31 u13 + `32 u23 + u33 = 1.
On solving the above we get
u11 = 3
u12 = 2 u13 = 7
`21 = 2/3 and u22 = 5/3 u23 = 11/3
`31 = 1 `32 = 6/5 u33 = 8/5.

1 0 0 3 2 7
Therefore, L = 2/3 1 0 , U = 0 5/3
11/3 . Therefore, (19) becomes
1 6/5 1 0 0 8/5

v1
LU X = B, i.e., LV = B, where V = U X. Let V = v2 . Then from LV = B we
v3
have:
1 0 0 v1 4
2/3 1 0 v2 = 5 .
1 6/5 1 v3 7
Multiplying and solving we get:
7 1
v1 = 4, v2 = , v3 = .
3 5
Again, from V = U X we have:

v1 4 3 2 7 x
v2 = 7/3 = 0 5/3 11/3 y
v3 1/5 0 0 8/5 z

4 3x + 2y + 7z
5
= 7/3 = 3
y 11
3
z .
8
1/5 5z
Comparing we get:

3x + 2y + 7z = 4;
5 11 7
y z =
3 3 3
8 1
z = .
5 5
Back substitution gives:
1 9 7
z= , y= , x= .
8 8 8

Home Work (Assignment)

Dr. Satish Shukla 9 of 20

(Q.1) Apply Crouts method to solve the equations:

3x + 2y + 7z = 4
2x + 3y + z = 5
3x + 4y + z = 7. Ans.x = 0.875, y = 1.125, z = 0.125.

(Q.2) Solve by factorisation method:

10x + y + z = 12
2x + 10y + z = 13
2x + 2y + 10z = 14. Ans.x = 1, y = 1, z = 1.

(Q.3) Apply Gauss elimination method to solve the equations:

x + 4y + z = 5
x + y 6z = 12
3x y z = 4. Ans.x = 1.647, y = 1.140, z = 2.084.

Solution of linear simultaneous equation (Iterative Methods).

Jacobis Method: Consider the following system of equations:

a11 x + a12 y + a13 z = b1 (20)

a21 x + a22 y + a23 z = b2 (21)
a31 x + a32 y + a33 z = b3 . (22)

We calculate the value of x from that equation in which coefficient of x has the largest
numerical value. Suppose, a11 a21 , a31 , then we calculate the value of x from equation
(20). Now we calculate the value of y from that equation (dont consider the equation
(20), because it has been used for calculating x) in which coefficient of y has the largest
numerical value. Suppose, a22 a32 , then calculate y from equation (21). Now we
remain with only one equation (22), therefore calculate z from equation (22). Then we
get:
1
x = (b1 a12 y a13 z) (23)
a11
1
y = (b2 a21 x a23 z) (24)
a22
1
z = (b3 a31 x a32 y) . (25)
a33

If y and z are known, then we can find the value of x from equation (23). Similarly,
if x and z are known, then we can find the value of y from equation (24) and if x and
y are known, then we can find the value of z from equation (25). But initially, all the
variables are unknown. Therefore, we start with an initial guess, say x = 0, y = 0 and
z = 0.
Dr. Satish Shukla 10 of 20

Step I. Now, putting y = z = 0 in (23) we find the first approximate value of x, we

write it x(1) . Similarly, by putting x = z = 0 in (24) we find the first approximate value
of y, we write it y (1) and by putting x = y = 0 in (25) we find the first approximate
value of z, we write it z (1) .
Step II. In next step, we put y = y (1) , z = z (1) in (23) and find the second approxi-
mate value of x, we write it x(2) . Similarly, by putting x = x(1) , z = z (1) in (24) we find
the second approximate value of y, we write it y (2) and by putting x = x(1) , y = y (1) in
(25) we find the second approximate value of z, we write it z (2) .
Repeat this process till we get the exact value of x, y and z upto the desired accu-
racy.

Formula for the Jacobis Method:

1 
x(n) = b1 a12 y (n1) a13 z (n1)

a11
1 
y (n) = b2 a21 x(n1) a23 z (n1)

a22
1 
z (n) = b3 a31 x(n1) a32 y (n1) .

a33

Example 6. Solve by Jacobis method:

20x + y 2z = 17
2x 3y + 20z = 25
3x + 20y z = 18.

Sol. Since the largest numerical value of coefficient of x in all equations is 20 (in the
first equation) therefore we calculate x from the first equation. Now we remain with the
second and third equations. In the second and third equations, the largest numerical
value of coefficient of y in remaining (second and third) equations is 20 (in the third
equation) therefore we calculate y from third equation. Now the z is calculated from
the second equation. Therefore, the formula for the Jacobis method will be:
1 
x(n) = 17 y (n1) + 2z (n1)

20
1 
y (n) 18 3x(n1) + z (n1)

=
20
1 
z (n) 25 2x(n1) + 3y (n1) .

=
20
Choose initial guess x(0) = y (0) = z (0) = 0.
Step I. From the above formula the first equation:
1 
x(1) = 17 y (0) + 2z (0) = 0.85

20
1 
y (1) 18 3x(0) + z (0) = 0.9

=
20
1 
z (1) 25 2x(0) + 3y (0) = 1.25.

=
20
Dr. Satish Shukla 11 of 20

Step II.
1 
x(2) = 17 y (1) + 2z (1) = 1.02

20
1 
y (2) 18 3x(1) + z (1) = 0.965

=
20
1 
z (2) 25 2x(1) + 3y (1) = 1.03.

=
20
Step III.
1 
x(3) = 17 y (2) + 2z (2) = 1.001

20
1 
y (3) 18 3x(2) + z (2) = 1.001

=
20
1 
z (3) 25 2x(2) + 3y (2) = 1.003.

=
20
Step IV.
1 
x(4) = 17 y (3) + 2z (3) = 1

20
1 
y (4) 18 3x(3) + z (3) = 1

=
20
1 
z (4) 25 2x(3) + 3y (3) = 1.

=
20
In the next step, we obtain x(5) = x(4) = 1, y (5) = y (4) = 1 and z (5) = z (4) = 1.
Therefore, x = 1, y = 1, z = 1 is the required solution.

The Gauss-Seidel Method. It is an improvement in the Jacobis method. In Ja-

cobis method, first we calculate x(1) , and when calculating y (1) we use the values x(0)
and z (0) , although the latest value of x which is x(1) is available at this stage. In the
Gauss-Seidel Method, while calculating y (1) , we use the latest available value of x, i.e.,
the value x(1) , instead x(0) . Similarly, in all the calculations we use the latest available
value of variables.
Formula for the Gauss-Seidel Method:
1 
x(n) = b1 a12 y (n1) a13 z (n1)

a11
1 
y (n) = b2 a21 x(n) a23 z (n1)

a22
1 
z (n) = b3 a31 x(n) a32 y (n) .

a33

Example 7. Find numerical solution of the system of equations given below by

Gauss-Siedel iteration method:
x + y + 54z = 110
27x + 6y z = 85
6x + 17y + 2z = 72.
Dr. Satish Shukla 12 of 20

Sol. Since the largest numerical value of coefficient of x in all equations is 27 (in the
second equation) therefore we calculate x from the second equation. Now we remain
with the first and third equations. In the second and third equations, the largest
numerical value of coefficient of y in remaining (first and third) equations is 15 (in the
third equation) therefore we calculate y from the third equation. Now the z is calculated
from the first equation. Therefore, the formula for the Gauss-Siedel iteration will be:
1 
x(n) = 85 6y (n1) + z (n1)

27
1 
y (n) = 72 6x(n) 2z (n1)

17
1 
z (n) = 110 x(n) y (n) .

54
Choose initial guess x(0) = y (0) = z (0) = 0.
Step I. From the above formula the first equation:
1 
x(1) = 85 6y (0) + z (0) = 3.1481

27
1 
y (1) = 72 6x(1) 2z (0) = 3.1242

17
1 
z (1) = 110 x(1) y (1) = 1.9208.

54
Step II.
1 
x(2) = 85 6y (1) + z (1) = 2.5250

27
1 
y (2) = 72 6x(2) 2z (1) = 3.1181

17
1 
z (2) = 110 x(2) y (2) = 1.9325.

54
Step III.
1 
x(3) = 85 6y (2) + z (2) = 2.5268

27
1 
y (3) = 72 6x(3) 2z (2) = 3.1161

17
1 
z (3) = 110 x(3) y (3) = 1.9325.

54
Step IV.
1 
x(4) = 85 6y (3) + z (3) = 2.5272

27
(4) 1 
72 6x(4) 2z (3) = 3.1159

y =
17
(4) 1 
110 x(4) y (4) = 1.9325.

z =
54
In the next step, we obtain x(4) x(5) = 2.5273, y (4) = y (5) = 3.1159 and z (4) = z (5) =
1.9325. Therefore, the solution is:
x = 2.5273, y = 3.1159 and z = 1.9325.
Dr. Satish Shukla 13 of 20

Home Work (Assignment)

(Q.1) What is the difference between Gauss and Gauss-Jordon elimination method.

(Q.2) What is the difference between Jacobis and Gauss-Siedel iteration method.

(Q.3) Solve by Gauss-Siedel iteration method:

x+y+z = 9
2x 3y + 4z = 13
3x + 4y + 5z = 40. Ans.x = 1, y = 3, z = 5.

Methods for Eigenvalue and Eigenvector of a Matrix

Eigenvalues and Eigenvectors: Let A be a square matrix of order n. Then, a matrix

X or order n 1 is called an eigenvector (or characteristic vector) of A, if there exists
a number such that AX = X. The number is called the eigenvalue (or charac-
teristic value) of A corresponding to the eigenvector X. There are n eigenvectors and
therefore n eigenvalues of a n n matrix. The numerically largest eigenvalue of A is
called the dominant eigenvalue of A and the eigenvector corresponding to the dominant
eigenvalue is called the dominant eigenvector of A.

The Power Method: By this method one can find out the dominant eigenvalue and
its corresponding eigenvector of a square matrix. Suppose A be a square matrix of
order n and X1 , X2 , . . . , Xn be the eigenvectors of A with corresponding eigenvalues
1 , 2 , . . . , n respectively. Suppose, 1 be the dominating eigenvalue of A. Then we
know that
AX1 = 1 X1 , AX2 = 2 X2 , . . . , AXn = n Xn . (26)
Suppose, a1 , a2 , . . . an are constants such that a1 6= 0, and:

v0 = a1 X1 + a2 X1 + + an Xn .

On multiplying by A we obtain

Av0 = A (a1 X1 + a2 X1 + + an Xn )
= a1 AX1 + a2 AX1 + + an AXn
= a1 1 X1 + a2 2 X1 + + an n Xn by (26).

Repetition of this process k times gives:

Ak v0 = a1 k1 X1 + a2 k2 X1 + + an kn Xn .

Therefore:
"  k  k #
2 n
Ak v0 = k1 a1 X1 + a2 X 1 + + an Xn .
k k
Dr. Satish Shukla 14 of 20

2
Since 1 is the dominating eigenvalue, we have k
, . . . , nk all are less than one, and
 k  k
2
therefore, if k is taken sufficiently large, the quantities k
, . . . , nk are negligible.
Therefore, we obtain from the above equation:

Ak v0 = k1 a1 X1 .

The above equation shows that as k increases, the value of Ak v0 approaches to a

multiple of the dominating eigenvector X1 of A.

Algorithm for Power Method.

(i) Choose an initial guess X0 for the dominating eigenvector X of A. Now

perform successive approximation for the value X by the following formula:

Xn = An X0 , n = 1, 2, . . . .

(ii) After each approximation perform scaling. Scaling is a process to convert

the vector into the unit vector, i.e., the vector v such that kvk = 1.

(iii) For scaling in power method, we use the maximum norm, and so, for scaling,
we divide the approximate value by its greatest (numerically) entry.

Example
 8. Apply power method to find the dominant eigenvalue of the matrix
5 4
A= .
1 2
 
1
Sol. Let X0 = be the initial guess. Then the first approximation:
1
       
5 4 1 9 1
X1 = AX0 = = =9 .
1 2 1 3 0.3333
Omitting the greatest entry 9, the second approximation:
       
5 4 1 6.3333 1
X2 = AX1 = = = 6.3333 .
1 2 0.3333 1.6667 0.2632
Omitting the greatest entry 6.3333, the third approximation:
       
5 4 1 6.0528 1
X3 = AX2 = = = 6.0528 .
1 2 0.2632 1.5264 0.2522
Omitting the greatest entry 6.0528, the fourth approximation:
       
5 4 1 6.0088 1
X4 = AX3 = = = 6.0088 .
1 2 0.2522 1.5044 0.2503
Omitting the greatest entry 6.0088, the fifth approximation:
       
5 4 1 6.0012 1
X5 = AX4 = = = 6.0012 .
1 2 0.2503 1.5006 0.2500
Dr. Satish Shukla 15 of 20

Omitting the greatest entry 6.0012, the sixth approximation:

       
5 4 1 6 1
X6 = AX5 = = =6 .
1 2 0.2500 1.5 0.2500
 
1
In the next step we obtain: X7 = 6 . From the above process, it is clear that
0.2500  
1
the dominating eigenvector of given matrix is X = and the corresponding
0.2500
eigenvalue is = 6.

Example 9. Apply power method to find the dominant eigenvalue of the matrix

2 1 0
A = 1 2 1 .
0 1 2

1
Sol. Let X0 = 1 be the initial guess. Then the first approximation:
1

2 1 0 1 1 1
X1 = AX0 = 1 2 1 1 = 0 = 1 0 .

0 1 2 1 1 1
Omitting the greatest entry 1, the second approximation:

2 1 0 1 2 1
X2 = AX1 = 1 2 1 0 = 2 = 2 1 .
0 1 2 1 2 1
Omitting the greatest entry 2, the third approximation:

2 1 0 1 3 0.75
X3 = AX2 = 1 2 1 1 = 4 = 4 1 .
0 1 2 1 3 0.75
Omitting the greatest entry 4, the fourth approximation:

2 1 0 0.75 2.5 0.7142
X4 = AX3 = 1 2 1 1 = 3.5 = 3.5 1 .
0 1 2 0.75 2.5 0.7142
Omitting the greatest entry 3.5, the fifth approximation:

2 1 0 0.7142 2.4285 0.7083
X5 = AX4 = 1 2 1 1 = 3.4285 = 3.4285 1 .
0 1 2 0.7142 2.4285 0.7083
Omitting the greatest entry 3.4285, the sixth approximation:

2 1 0 0.7083 2.4167 0.7073
X6 = AX5 = 1 2 1 1 = 3.4166 = 3.4166 1 .
0 1 2 0.7083 2.4167 0.7073
Dr. Satish Shukla 16 of 20

Omitting the greatest entry 3.4166, the seventh approximation:

2 1 0 0.7073 2.4146 0.7071
X7 = AX6 = 1 2 1 1 = 3.4146 = 3.4146 1 .
0 1 2 0.7073 2.4146 0.7071

0.707
Therefore, the dominant eigenvector: X = X6 = X7 = 1 and the correspond-
0.707
ing eigenvalue is = 3.414. (correct upto three places of decimals).

Example 10. Apply power method to find the dominant eigenvalue of the matrix

1 1 0
A = 1 2 1 .
0 1 1

1
Sol. Let X0 = 1 be the initial guess. Then the first approximation:
1

1 1 0 1 2 0.5
X1 = AX0 = 1 2 1 1 = 4 = 4 1 .
0 1 1 1 2 0.5
Omitting the greatest entry 4, the second approximation:

1 1 0 0.5 1.5 0.5
X2 = AX1 = 1 2 1 1 = 3 = 3 1 .
0 1 1 0.5 1.5 0.5
Omitting the greatest entry 3,
the third approximation:

1 1 0 0.5 1.5 0.5
X3 = AX2 = 1 2 1 1 = 3 = 3 1 .
0 1 1 0.5 1.5 0.5

0.5
Therefore, the dominant eigenvector: X = X2 = X3 = 1 and the corre-
0.5
sponding eigenvalue is = 3.

QR-Method.
Rotation Matrix. An orthogonal matrix P (r, s) = [pij ]nn , where r < s is called a
rotation or rotation matrix if it all of its entries are same as the identity matrix with
exception that
prr = pss = cos and prs = psr = sin .
Dr. Satish Shukla 17 of 20

The quantity is called the rotation, i.e.,

1
1
..

.


cos sin



P (r, s) = . .

.


sin cos


...


1
1

where the trigonometric functions are in the r-th and s-th columns and all other entries
are zero. If r > s we define P (r, s) = [P (s, r)]T = [P (s, r)]1 .

Example 11. In two dimensions, for the rotation of the axes through an angle
in anti-clockwise direction, the rotation matrix or rotaion is given by:
 
cos sin
P = P (1, 2) = .
sin cos

Example 12. In three dimensions for the rotation about X-axis through an angle
in anti-clockwise direction, the rotation matrix is given by:

1 0 0
Px = P (2, 3) = 0 cos sin .
0 sin cos

Observation. Pre-multiplication of P (2, 3) to any arbitrary matrix A affects only

the 2nd and 3rd rows. Indeed:

2nd row of P (2, 3)A = cos (2nd row of A) sin (3rd row of M )
3rd row of P (2, 3)A = sin (2nd row of A) + cos (3rd row of M ).

The above result is true for any arbitrary rotation.

QR factorization. Suppose A be a matrix. Then the process in which we find two

matrices Q and R such that A = QR, where Q is an orthogonal matrix (i.e., Q1 = QT )
and R is an upper triangular matrix; is called the QR factorization of the matrix A.
In this case, A = QR is called a QR representation of A.
The QR-algorithm. Suppose, A be a given matrix. Starting with initial value A1 = A,
define a sequence An by the pair:

An = Qn Rn and An+1 = Rn Qn
Dr. Satish Shukla 18 of 20

where each Qn is an orthogonal matrix and each Rn is an upper triangular matrix.

If this sequence converges, then lim An = D, where D is a diagonal matrix and the
n
entries of the diagonal of D are the eigenvalues of A.
How to apply the QR-method. In this method, the main task is the QR-factorization
of the matrix whose eigenvalues are to be detrained. For this, note that if A = QR
is the QR-factorization os A, then QT A = R, i.e., QT A is an upper triangular matrix.
Therefore, we try to find an orthogonal matrix QT such that QT A is an upper triangular
matrix.
The next question aries that how to find the orthogonal matrix QT such that QT A
is an upper triangular matrix? Then, we reduce the matrix A into an upper triangular
matrix by vanishing (making zero) its all the entries below the diagonal one by one.
For this, in first step, we pre-multiply the matrix A = [ars ]nn by the rotation matrix
P (2, 1) which vanishes the entry a21 (obviously, if a21 is already zero, we omit this
step). In next step, we pre-multiply the matrix P (2, 1)A by a the rotation matrix
P (3, 1) which vanishes the entry a31 (obviously, if a31 is already zero, we omit this
step). We repeat this process till we get all the entries below the diagonal vanished.
Thus, we find:

P (n, n 1) P (3, 1)P (2, 1)A = R (an upper triangular matrix)

or
QT A = R
where QT = P (n, n 1) P (3, 1)P (2, 1). Now, we note that QT A1 = QT A = R =
QT A1 Q = RQ = A2 . Thus, we have found the first iteration of the QR-method. Now,
we repeat the whole process till we get an upper triangular matrix An . The diagonal
entries of An are the eigenvalues of A.
The eigenvalue shifting. The QR-iteration is a very lengthy process. To decrease the
necessary number of iteration we use the shifting of eigenvalues. For this, we shift the
eigenvalues of A by , i.e., we use A I instead A in the QR-method, where is
chosen such that it is not so far from the actual eigenvalue of A. It is not a hard-n-fast,
but one can choose as the mean of diagonal of the matrix A.

Important. If we want to vanish ars , i.e., the entry at position rth row and
j th column of the matrix A, then we pre-multiply the matrix A by the rotation
P (r, s). To find the rotation P (r, s) we have to find the values of cos and sin
in the expression of P (r, s) and these values can be obtained by:
ass ars
cos = p , sin = p .
a2ss 2
+ ars ass + a2rs
2

4 2 1
Example 13. Find the eigenvalues of A = 0 1 0 by QR-method.
0 2 3
Dr. Satish Shukla 19 of 20

4+1+3
Sol. Let = 3
= 2.6667 3. Then, shifting the eigenvalues by we get:

1 2 1
A 3I = 0 2 0 .
0 2 0

To convert the matrix A 3I into a diagonal matrix we have to vanish all the elements
below the diagonal matrix. Since a21 , a22 are already zero in A 3I, we move to the
entry a32 = 2. To vanish a32 = 2 we pre-multiply A 3I by the rotation P (3, 2), where

1 0 0
P (3, 2) = [P (2, 3)]T = 0 cos sin , where
0 sin cos

a22 2 1
cos = p =p = ,
a222 + a232 (2)2 + 22 2
a32 2 1
sin = p =p = .
a222 + a232 (2)2 + 22 2

1 0 0 2 0 0
Therefore, P (3, 2) = 0 1/2 1/ 2 = 1 0 1 1 . Now,
2
0 1/ 2 1/ 2 0 1 1

2 0 0 1 2 1 2 2 2 2
1 1
P (3, 2) (A 3I) = 0 1 1 0 2 0 = 0 4 0
2 0 1 1 0 2 0 2 0 0 0

which is an upper triangular matrix. Therefore, QT = P (3, 2), and so,

A2 = QT (A 3I)Q = P (3, 2)(A 3I)[P (3, 2)]T

2 2 2 2 2 0 0
1 1
= 0 4 0 0 1 1
2 0 0 0 2 0 1 1

1 2/2 3 2/2
= 0 2 2 .
0 0 0

Therefore, A2 is an upper triangular matrix and so eigenvalues of A 3I are the

diagonal entries of A2 , i.e., 1, 2, 0. Thus, the eigenvalues of A are 4, 1, 3.

Note. It is not possible to calculate the

 eigenvalues
 of an arbitrary matrix with
0 1
QR-method. For instance, if A = , then the QR-method does not
1 0
converge. More interestingly, if we shift the eigenvalues by 1, then QR-method
works, and converges in only one step.
Dr. Satish Shukla 20 of 20

 
0 1
Example 14. Find the eigenvalues of A = . Take a shift of eigenvalues
1 0
by 1 and use QR-method.

Sol. Let = 1. Then, shifting the eigenvalues by we get:

 
1 1
A (1)I = A + I = .
1 1

To convert the matrix A + I into a diagonal matrix we have to vanish all the elements
below the diagonal matrix. To vanish a21 = 1 we pre-multiply A + I by the rotation
P (2, 1), where  
T cos sin
P (2, 1) = [P (1, 2)] =
sin cos
where
a11 1 1
cos = p = = ,
a211 + a221 12
+1 2 2
a21 1 1
sin = p = = .
a211
+ a221 2
1 +1 2 2
   
1/ 2 1/2 1 1 1
Therefore, P (2, 1) = = 2 . Now,
1/ 2 1/ 2 1 1
     
1 1 1 1 1 1 2 2
P (2, 1) (A + I) = =
2 1 1 1 1 2 0 0

which is an upper triangular matrix. Therefore, QT = P (2, 1), and so,

A2 = QT (A + I)Q = P (2, 1)(A + I)[P (2, 1)]T

   
1 2 2 1 1 1
=
2 0 0 2 1 1
 
2 0
= .
0 0

Therefore, A2 is an upper triangular matrix, and so, the eigenvalues of A + I are the
diagonal entries of A2 , i.e., 2, 0. Thus, the eigenvalues of A are 1, 1.