Num Linear

Review of matrices
Let m, n IN. A rectangle of numbers written like

a11 a12 . . . a1n
a21 a22 . . . a2n
A= .

.. .. ..
.. . . .
am1 am2 . . . amn
where each aij IR is called a matrix with m rows and n columns or an

m n matrix.
The above matrix may be denoted by [aij ]1im (or just by [aij ]).
1jn
The number aij is called the (i, j) entry. It lies in the i th row and j th
column.
Dana Mackey (DIT) Numerical Methods 1 / 56

The entries like a11 , a22 , a33 and so on are called the diagonal entries.
These entries form the main diagonal. Notice that the main diagonal
finishes in the bottom right corner only if the matrix has the same number
of rows as columns, i.e. we have a square matrix.
A matrix composed of only one column (that is an m 1 matrix) is called

a column matrix or vector.
Two matrices are equal if and only if they have the same size and the
corresponding entries are all equal.
For each m and n in IN, we use Mm,n for the set of all m n matrices.
The m n matrix whose entries are all zero is called the m n zero
matrix and is denoted 0m,n or simply 0.

Matrix addition
If A = [aij ] and B = [bij ] are two matrices in Mm,n then we define

A + B Mm,n by

a11 + b11 a12 + b12 ... a1n + b1n
a21 + b21 a22 + b22 ... a2n + b2n
A + B = [aij + bij ] =

.. .. .. ..
. . . .
am1 + bm1 am2 + bm2 . . . amn + bmn

1 2 1 0 3 2 1 1 3
For example, + = .
4 3 0 5 4 1 9 1 1
Note: We can only add two matrices if they are of the same size.

Scalar multiplication
If A = [aij ] Mm,n and k IR then we define the product of A with the

scalar number k to be kA Mm,n given by

ka11 ka12 ... ka1n
ka21 ka22 ... ka2n
kA = [kaij ] = .

.. .. ..
.. . . .
kam1 kam2 . . . kamn
This operation is called scalar multiplication. Note that

kA = [kaij ] = [aij k] so the notation Ak will also be used, and then kA = Ak.

1 2 5 10 1 2 0 0
For example 5 = and 0 = .
4 3 20 15 4 3 0 0

Inner product
The next question is how to multiply two matrices. We start with a special
case.
Let n IN and let A be a 1 n matrix (a row matrix) and let Bbe an

b1
b2
n 1 column matrix. That is A = a1 a2 . . . an while B = . . We

..
bn
define the (inner) product of A and B to be

b1
b2

AB = a1 a2 . . . an . = a1 b1 + a2 b2 + + an bn .

..
bn

Note that the inner product is of a row and a column of the same length
(the length of a row or column matrix is the number of entries in it) and
that the answer is a real number (not a matrix).
For example

2
2 1 3 5 = 2(2) + (1)5 + 3(3) = 4 5 + 9 = 0.
3

Matrix multiplication
Let m, n and p IN. Suppose A Mm,n and B Mn.p . Then we define

the product of A and B to be the m p matrix formed as follows:
The (i, j) entry of AB is the inner product of the i th row of A with the j th
column of B.

1 3 2 1
2 0 1
Example: Let A = and B = 0 4 2 1 . Then A M2,3
3 1 1
2 0 2 3
and B M3,4 implies AB M2,4 and

1 3 2 1
2 0 1 0 6 2 1
AB = 0 4 2 1 =
3 1 1 5 5 10 7
2 0 2 3

In general AB 6= BA for matrices
In order to multiply two matrices, A by B, the number of columns of A
must equal the number of rows of B. In other words, if A Mm,n and
B Mp,q then to form AB we must have that n = p. To form BA we
must have q = m.
In particular we note that AB 6= BA in this case. Even if we take two

matrices A and B so that AB and BA are the same
size, we cannot

1 2 5 6
assume that AB = BA. For example, if A = and B = then
3 4 7 8

1 2 5 6 19 22
AB = =
3 4 7 8 43 50
while
5 6 1 2 23 34
BA = = 6 AB.
=
7 8 3 4 31 46

Elementary Row operations
Given a matrix, the following operations on the rows of that matrix are
called (elementary) row operations.
1 Interchange rows, e.g.

0 1 2 3 1 3
1 R2
3 1 3 R 0 1 2
2 9 4 2 9 4
2 Multiply a row by a non-zero real number
1 13 1

3 1 3 1
R 3 R1
0 1 2 1 0 1 2
2 9 4 2 9 4
3 Add a scalar multiple of one row to another
1
1 13 1 1 13

1 3 1 1
R3 R3 2R1
0 1 2 0 1 2 = 0 1 2
2
2 9 4 2 2(1) 9 3 4 2(1) 0 8 31 6
Echelon form
A matrix A = [aij ] is said tobe in echelon form if aij = 0 for i > j. For
1 2 2 3
example, 0 2 3 17 is in echelon form. A square matrix in
0 0 5 1
echelon form is called upper triangular.
Any matrix can be converted to echelon form by using elementary row

operations. The steps involved are as follows. Let A = [aij ] Mm,n .

1 Look at the first column. We want all entries below the first one, a11 ,
to be zero. If necessary (that is, if a11 = 0), interchange two rows so
that a11 6= 0.
2 Add multiples of the first row to the other rows to ensure a21 , a31 ,
. . . , am1 are all zero. For example, add a
a11 row 1 to row 2, etc.
21
3 Move to the second column. We want all the entries below a22 , that
is a32 , a42 , etc., to be zero. If necessary, interchange two rows (not
the first row) so that a22 6= 0.
4 Add multiples of the second row to ensure a32 , a42 , . . . , am2 are all
zero.
5 Move along the columns in this way until the matrix is in echelon
form.

Example:

0 1 2
Convert the matrix 3 1 3 to echelon form.
2 9 4

Systems of linear equations
A linear equation in one variable is an equation of the form
ax = b
where a and b are given real numbers, and x is a variable.
A linear equation in n variables is an equation of the form
a1 x1 + a2 x2 + + an xn = b
where a1 , . . . , an , b IR. For example, 2x1 + 3x2 = 7 is a linear equation in

two variables; 2x 3y + 4z = 5 is a linear equation in three variables.

We can also consider a system of m equations in n variables. That is,
a11 x1 + a12 x2 + + a1n xn = b1

a21 x2 + a22 x2 + + a2n xn = b2
.. ..
. .
am1 x1 + am2 x2 + + amn xn = bm .
The system of m equations above can also be written as one matrix

equation:
a11 a12 . . . a1n x1 b1
a21 a22 . . . a2n x2 b2
.. .. = ..

.. .. ..
. . . . . .
am1 am2 . . . amn xn bm
and we will usually write this as AX = B. When written in this form, a
solution of the system will be a n 1 column matrix with n elements (or
n-vector). A linear system may have no solutions, many solutions or just a
unique solution.
The method of Gaussian Elimination for solving systems
Suppose the system is AX = B where A Mm,n and B Mm,1 . Form the

.
augmented matrix [A .. B] and convert this matrix to echelon form to get a
.
matrix [A0 .. B 0 ]. The echelon system A0 X = B 0 has the same solutions as
the original system AX = B but is much easier to solve. This process of
solving a linear system is called Gaussian elimination.
Example: Solve the system of linear equations,
x2 + 2x3 = 3
x1 + 2x2 + 3x3 = 2
x1 2x2 2x3 = 0

More examples
Solve, by Gaussian elimination, the following systems of linear equations.
x1 2x2 2x3 = 4
(i) 3x1 x2 x3 = 7
6x1 +x2 +x3 = 5.
x1 +x2 +x3 x4 =0
3x1 x2 x3 +2x4 =0
(ii)
4x1 +5x2 +x3 x4 =0
9x1 +6x2 +2x3 x4 =0

Gaussian Elimination with partial pivoting
Example: Consider the following system

2 2 1 43
x1 + x2 + x3 =
3 7 5 15
1 1 1 5
x1 + x2 x3 = (1)
3 7 2 6
1 3 2 12
x1 x2 + x3 =
5 7 5 5
Exercise: Using Gaussian elimination and back substitution show that the
exact solution is x1 = 1, x2 = 7 and x3 = 1.

Now write the system using four decimal digit rounding arithmetic
0.6667x1 + 0.2857x2 + 0.2000x3 = 2.867

0.3333x1 + 0.1429x2 0.5000x3 = 0.8333 (2)
0.2000x1 0.4286x2 + 0.4000x3 = 2.400
Remarks:
1 To obtain the 4-digit floating point approximation of a number, we

use the rounding method (e.g. 23 = 0.66666 . . . gives 0.6667.)
2 Whenever performing a calculation (e.g. addition or multiplication)
involving two or more numbers, the 4-digit rounded form is used for
each number and then the result of the calculation is rounded as well.
For example, 0.1234 1.567 = 0.193368 . . . = 0.1934. This is the
source of roundoff error accummulation.
3 When performing row operations to get the echelon form, the entries
below the pivot are implicitly set equal to zero, as opposed to being
explicitly calculated, in order to avoid unnecessary calculations.
The first step of the Gaussian elimination yields
0.6667x1 + 0.2857x2 +0.2000x3 = 2.867

+ 0.0001x2 0.6000x3 = 0.5997 (3)
0.5143x2 +0.3400x3 = 3.260
where the row operations performed were

0.3333
R2 R1 = R2 0.4999 R1
0.6667
0.2000
R3 R1 = R3 0.3 R1
0.6667
and the above entries have been calculated as follows:
0.1429 0.2857 0.4999 = 0.1429 0.1428 = 0.0001

0.8333 2.867 0.4999 = 0.8333 1.433 = 0.5997
0.4286 0.2857 0.3 = 0.4286 0.08571 = 0.5143
and so on.
The next step of the Gaussian elimination gives
0.6667x1 + 0.2857x2 +0.2000x3 = 2.867

+ 0.0001x2 0.6000x3 = 0.5997 (4)
3086x3 = 3087
which gives the solution:
x3 = 1.000 (exact)
x2 = 3.000 (relative error=57%)
and x1 = 2.715 (relative error=200%)!

Remark:
The large errors obtained in the previous example were obtained because of
a cancellation (or loss of significance) error which occurred while working
on the first column. This introduced a small pivot, 0.0001, in the second
column which amplified the error. We cannot always avoid cancellation
errors but at least we can try to avoid the use of very small pivots.

The partial pivoting strategy is the simplest scheme which eliminates
the use of small pivots.
For each column j let

Mi = max |Aij |
ji
and let i0 be the smallest value of i (i > j) for which this maximum is
achieved. Then interchange rows j and i0 .
In other words, while working on column j, look for the entry with the
largest absolute value below the pivot (Aij with i j). If this largest entry
appears in more than one row, take the one closest to the pivot row, i0 ,
and swap it with the current pivot row, j.
Exercise: Apply the technique of Gaussian elimination with partial

pivoting to system (2) and verify if the error has improved.

0.6667 0.2857 0.2000 2.867
R2 0.4999 R1
0.3333 0.1429 0.5000 0.8333 ( )
R3 0.3 R1
0.2000 0.4286 0.4000 2.400

0.6667 0.2857 0.2000 2.867
0 0.0001 0.6000 0.5997 (R2 R3 )
0 0.5143 0.3400 3.260

0.6667 0.2857 0.2000 2.867
0 0.5143 0.3400 3.260 (R3 0.0001944 R2 )
0 0.0001 0.6000 0.5997

0.6667 0.2857 0.2000 2.867
0 0.5143 0.3400 3.260
0 0 0.5999 0.603
Partial pivoting gives the solution x3 = 1.001, x2 = 7.000 and x1 = 1.000.

Exercise:
Consider the following system
3x1 + x2 + 4x3 x4 = 7
2x1 2x2 x3 + 2x4 = 1
5x1 + 7x2 + 14x3 8x4 = 20
x1 + 3x2 + 2x3 + 4x4 = 4
1 Show, by direct substitution, that the exact solution is (1,-1,1,-1).

2 Solve the system using Gaussian elimination without pivoting and
calculate the relative errors.
3 Solve the system using Gaussian elimination with partial pivoting and
calculate the relative errors.

Scaled partial pivoting
Consider now the example
0.7x1 + 1725x2 = 1739

0.4352x1 5.433x2 = 3.271
The exact solution is x1 = 20 and x2 = 1. Partial pivoting would leave the

equations unchanged since 0.7 is the largest entry in the first column.
Gaussian elimination then gives
0.7x1 + 1725x2 = 1739

1077x2 = 1078
hence x2 = 1.001 and x1 = 17.14.

The large errors obtained in this case are due to the fact that the entries
0.7 and 0.4352 are not compared with the other coefficients in their
corresponding rows. For instance, 0.7 > 0.4352 but
0.7 0.4325
<
1725 5.433
If, instead, we choose the pivot which is largest in magnitude relative to

the other coefficients in the equation, we need to swap the rows and use
0.4352 as pivot
0.4352x1 5.433x2 = 3.271

0.7x1 + 1725x2 = 1739
which gives x2 = 1.000 and x2 = 20.000.

Scaled partial pivoting
Construct a scale vector as follows
Si = max |aij |, for each 1i n

1jn
(Si is the largest absolute value in row i.)
During step i of the Gaussian elimination (column i), let

|aji |
Mi = max
ijn Sj
and let j0 be the smallest j for which this maximum occurs. If j0 > i then
we interchange rows i and j0 .
Note: The scale vector S does not change during the Gaussian
elimination process.

Exercise:
Consider the system
3x1 + x2 + 4x3 x4 = 7
2x1 2x2 x3 + 2x4 = 1
5x1 + 7x2 + 14x3 8x4 = 20
x1 + 3x2 + 2x3 + 4x4 = 4
Recall that the exact solution of this system is (1,-1,1,-1). Calculate the
solution using Gaussian elimination with scaled partial pivoting and
compare the errors with those obtained using (simple) partial pivoting.

The LU decomposition method
Consider the system of equations, written in matrix form
AX = B (5)
where A Mn,n is a square matrix and X , B Mn,1 .
The LU decomposition (or factorisation) of the matrix A consists of

writing the matrix A as the product of a lower triangular matrix L and an
upper triangular matrix U, such that A = LU. (Recall that a matrix is
called upper triangular if all entries below the main diagonal are zero and
lower triangular if all entries above the main diagonal are zero.)

The procedure of solving the system of equations (5) using LU
decomposition is the following.
Step 1: Determining the LU decomposition The matrices L and U can

be determined by writing (in the case of a 3 3 matrix, for example)

a11 a12 a13 1 0 0 u11 u12 u13
A = a21 a22 a23 = L U = l21 1 0 0 u22 u23
a31 a32 a33 l31 l32 1 0 0 u33
and solving this system for the entries lij and uij .

Step 2: Solving the LU decomposed system
After the LU factorisation, system (5) becomes
LUX = B
which can be solved as two successive problems:
(1) LY = B; (2) UX = Y .
Each of these two systems is easy to solve since the matrix of coefficients
is either upper or lower triangular.

Example:
Solve the following system using the LU decomposition method.
x1 + x2 x3 = 1
x1 + 2x2 2x3 = 5
2x1 + x2 + x3 = 10

Properties of the LU decomposition method:
1: Not every matrix has an LU-decomposition. For example,

0 0 1
A = 0 1 0
1 0 1
cannot be written as the product of a lower triangular and upper triangular

matrices.
However, given any invertible matrix A, it is always possible to rearrange

the rows of A so that the resulting matrix does have an LU-decomposition.
A sufficient condition for a matrix A to have an LU decomposition is that

Gaussian elimination can be performed on A without row interchanges.

2. When a matrix has an LU-decomposition, that decomposition is not
unique. Given the factorisation A = LU, if we define the new matrices
L0 = LD and U 0 = D 1 U
where D is any invertible diagonal matrix, then L0 is lower triangular, U 0 is

upper triangular and A = L0 U 0 so we found another LU-factorisation.
In fact, once a matrix A admits an LU-factorisation, there are an infinite

number of choices for the matrices L and U!

3. The LU-decomposition is unique up to scaling by a diagonal matrix.
This means that, if A admits two LU-factorisations,
A = L1 U1 = L2 U2
then we must have
L1 = L2 D and U2 = DU1
for some diagonal matrix D.
Note: A product of upper (or lower) triangular matrices is also an upper

(or lower) triangular matrix.

4. If A admits a LU-factorisation then we can find matrices L and U such
that L has 1s along the diagonal. (Similarly, we can find a decomposition
in which U has 1s along the diagonal.)
5. The decomposition A = LU is unique if we require that L (or U) has 1s

along its diagonal!

LU Factorization using Gaussian Elimination
Consider the following example

1 4 5 1 4 5 1 4 5
A = 4 20
32 = 0 4 12 = 0 4 12
5 32 64 0 12 39 0 0 3
where the row operations used were as follows
R2 R2 4R1
R3 R3 5R1 R3 R3 3R2
On the other hand, the LU factorization of the matrix A is

1 4 5 1 0 0 1 4 5
A = 4 20 32 = 4 1 0 0 4 12 = LU
5 32 64 5 3 1 0 0 3

We notice that the upper triangular echelon form of the matrix A yields
the matrix U in the LU decomposition.
The matrix L is constructed as a lower triangular matrix with 1s along the

main diagonal and the negatives of the Gaussian elimination multipliers in
the appropriate positions.
An algorithm for constructing an LU decomposition for a matrix A can be

constructed as follows.
Initialize the matrix L as the identity matrix and U as the matrix A.

Perform Gaussian elimination on the matrix U and insert the row
multipliers into the appropriate positions in the matrix L.
When the echelon form for U has been achieved, L will be a lower
triangular matrix. The LU decomposition is complete.

Example
Find the LU factorization of the matrix

4 12 8 4
1 7 18 9
A= 2 9

20 20
3 11 15 14
Gaussian elimination:

1 0 0 0 4 12 8 4
0 1 0 0 1 7 18 9
L= 0 0 1 0
U =2 9 20 20

0 0 0 1 3 11 15 14

1 0 0 0 4 12 8 4
1/4 1 0 0 0 4 16 8 R2 14 R1
L= U =
1/2 0 1 0 0 3 16 18 R3 21 R1
3/4 0 0 1 0 2 9 11 R4 43 R1

1 0 0 0 4 12 8 4
1/4 1 0 0 0 4 16 8
L= U =
1/2 3/4 1 0 0 0 4 12 R3 34 R2
3/4 1/2 0 1 0 0 1 7 R4 21 R2

1 0 0 0 4 12 8 4
1/4 1 0 0 0 4 16 8
L=
1/2
U =
3/4 1 0 0 0 4 12
3/4 1/2 1/4 1 0 0 0 4 R4 14 R3
which completes the LU decomposition.

Exercise: Check that A = LU!

Why does this work?
Consider the matrices

1 0 0 1 0 0
M1 = 2 1 0 , M2 = 0 1 0 .
3 0 1 0 4 1
Exercise 1: Show that multiplying a matrix A by the matrix M1 has the

same effect as performing the row operations R2 R2 + 2R1 and
R3 R3 + 3R1 , while multiplication by the matrix M2 has the same effect
as performing the row operation R3 R3 + 4R2 .
Exercise 2 Calculate the inverse matrices M11 , M21 and the product
M11 M21 .

Remarks
1 The Gaussian elimination procedure can be viewed as repeated

multiplication by lower triangular matrices and the inverse of their
product yields the matrix L in the LU decomposition.
2 Note that we have avoided row interchanges in the previous examples.
If such row operations are needed (for example when performing
Gaussian elimination with pivoting), the procedure is slightly different.
(We wont cover this case now!)
3 The old procedure is referred to as LU decomposition by direct
calculation or Doolittles method.

The Inverse of a Matrix
Let A Mn,n . If there exists B Mn,n such that AB = I = BA then we say
A is invertible and B is an inverse of A. We write B = A1 .
Note that if B is an inverse of A, then A is an inverse of B.
Procedure to invert a matrix:

row operations to invert a 2 2 invertible matrix.
We can use elementary

1 2
Start with A = and augment the identity matrix to form the
3 4
2 4 matrix
..
1 2 . 1 0 .
.
3 4 .. 0 1
Now use elementary row operations to convert the left hand side to the
identity matrix but performing the row operations across the entire row.

When the left hand matrix becomes equal to the identity, the right hand
matrix will be the inverse of A.
Example:

1 2 1 0 R2 R2 3R1 1 2 1 0 R1 R1 +R2 1 0 2 1

3 4 0 1 0 2 3 1 0 2 3 1
1

R2 2 R2 1 0 2 1
.
0 1 2 12
3

1 2 2 1
Hence the inverse of is 3 .
3 4 2 21
This procedure can be used to invert any n n matrix, if it is invertible.

0 1 2
Example: Find the inverse of the matrix A = 1 2 3 .
1 2 2

2 2 1
The answer is A1 = 1 2 2 . Check that AA1 = I = A1 A.
0 1 1
Note: A square matrix is invertible if and only if it does not contain a zero
on the main diagonal after it has been reduced to echelon form by
elementary row operations.

1 2 3
For example, the matrix A = 3 1 5 has no inverse!
2 3 2

Solving systems of linear equations
The inverse of a matrix gives a new method for solving a system of linear
equations:
If AX = B then X = A1 B
provided A is invertible.
Exercise:
Solve the following system of equations by inverting the matrix of
coefficients.
x + 2y + 3z = 1
x y + 4z = 4
2x 2y + z = 8

Iterative methods for systems of linear equations
Linear systems arising from practical applications tend to be large and

their coeffficient matrices will often be sparse (that is, only a small
percentage of entries are non-zero). For such systems, iterative methods
are generally more efficient than direct methods.
Basic Idea:
Iterative methods for systems of linear equations are similar to the

algorithms for finding roots of nonlinear algebraic equations.
For example, consider the linear system
AX = B (6)
where A is an n n matrix and B is an n-vector.

This can be converted to a fixed-point problem:
X = TX + C
for some (new) n n matrix T , called the iteration matrix, and n-vector
C.
An iterative method for solving this problem consists of starting with a

suitable initial guess, X (0) and then constructing the iterative sequence
X (k) as follows
X (k+1) = TX (k) + C , k = 1, 2, . . .
This algorithm is terminated when two successive vectors X (k) and X (k+1)
are close enough in a sense which remains to be defined.

Definition:
Let A be an n n matrix. If we can write A = M N, where M and N are

n n matrices and M is invertible, then the pair (M, N) is called a
splitting of the matrix A.
If we have a splitting A = M N in Equation (1) above then the system

can also be written as
MX = NX + B or X = M 1 NX + M 1 B = TX + C
where we let T = M 1 N and C = M 1 B.

1. The Jacobi method
This method is based on splitting the coefficient matrix A as follows:
A = D (L + U)
where D is the diagonal part of A, while L and U are the strictly lower
and upper triangular parts of A, respectively. (So, according to the
definition above, we have M = D and N = L + U.)
Note that L and U in this context are not related at all to the matrices L,
U we would get from a LU-decomposition of A.
Example: Find the Jacobi splitting for the matrix

5 1 2
A = 3 9 4
1 2 7

Recall, from the definition of a splitting, that the matrix M has to be
invertible. In the case of the Jacobi splitting this means that the diagonal
part of A has to be invertible so all diagonal elements have to be non-zero.
If at least one diagonal element of A happens to be zero, the equations in
the system must be reordered so that the zero entry is relocated to a
non-diagonal position.
The Jacobi method iteration formula
X (k+1) = TX (k) + C
where T = D 1 (L + U) and C = D 1 B.

Exercise 1:
Consider the following system of equations
5x1 + x2 + 2x3 = 10
3x1 + 9x2 + 4x3 = 14
x1 + 2x2 7x3 = 33.
Show that the exact solution is x1 = 1, x2 = 3, x3 = 4 and that the
iteration sequence associated with the Jacobi method is given by
(k+1) 1 (k) (k)
x1 = [10 x2 2x3 ]
5
(k+1) 1 (k) (k)
x2 = [14 + 3x1 4x3 ]
9
(k+1) 1 (k) (k)
x3 = [33 x1 2x2 ]
7
Find an approximate solution such that

n o
(k) (k) (k)
max |x1 x1 |, |x2 x2 |, |x3 x3 | 0.01
Solution
(k) (k) (k) (k) (k) (k)

k x1 x2 x3 x1 1 x1 + 3 x1 4
0 0 0 0 -1 3 -4
1 2.0000 -1.5556 4.7143 1.0000 1.4444 0.7143
2 0.4253 -2.9841 4.5555 -0.5746 0.0159 0.5555
3 0.7746 -3.4384 3.9224 -0.2254 -0.4384 -0.0775
4 1.1187 -3.0407 3.8425 0.1187 -0.0407 -0.1575
5 1.0711 -2.8904 4.0053 0.0711 0.1095 0.0053
6 0.9759 -2.9787 4.0415 -0.0240 0.0213 0.0415
7 0.9791 -3.0264 4.0027 -0.0208 -0.0264 0.0027
8 1.0042 -3.0081 3.9895 0.0042 -0.0081 -0.0105
9 1.0058 -2.9939 3.9983 0.0058 0.0061 -0.0017

2. The Gauss-Seidel method
(k+1) (k+1)
An obvious improvement in the Jacobi method is to use x1 and x2
as soon as they have been calculated rather than wait for the next
iteration. This modification yields the Gauss-Seidel iteration algorithm:
(k+1) 1 (k) (k)

x1 = [10 x2 2x3 ]
5
(k+1) 1 (k+1) (k)
x2 = [14 + 3x1 4x3 ]
9
(k+1) 1 (k+1) (k+1)
x3 = [33 x1 2x2 ]
7
Exercise 2: From the approximation equations above, show that the

Gauss-Seidel method is based on the splitting
A = (D L) U
where D, U and L have the same meaning as in the Jacobi method.

Exercise 3 : Redo Exercise 1 above using the Gauss-Seidel method.
(k) (k) (k) (k) (k) (k)

k x1 x2 x3 x1 1 x1 + 3 x1 4
0 0 0 0 -1 3 -4
1 2.0000 -0.8889 4.7460 1.0000 2.1111 0.7460
2 0.2794 -3.5718 3.7337 -0.7206 -0.5718 -0.2663
3 1.2209 -2.8080 4.0864 0.2209 0.1920 0.0864
4 0.9270 -3.0627 3.9716 -0.0730 -0.0627 -0.0283
5 1.0239 -2.9794 4.0093 0.0239 0.0205 0.0092
6 0.9922 -3.0067 3.9969 -0.0078 -0.0067 -0.0030
We have obtained the same order of accuracy after only 6 steps therefore
the Gauss-Seidel method is faster than the Jacobi method.

Convergence properties for the Jacobi and Gauss-Seidel
methods
A matrix is called strictly diagonally dominant if, for each row, the
absolute value of the diagonal element is strictly larger than the sum
of absolute values of the other elements.
If A is strictly diagonally dominant then both the Jacobi and
Gauss-Seidel algorithms will converge for any choice of the initial
vector approximation!
Note that strict diagonal dominance is a sufficient condition: it does
guarantee convergence but convergence might still occur even if the
condition is not satisfied.

Num Linear

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Num Linear

Hochgeladen von

Copyright:

Verfügbare Formate

Review of matrices

Let m, n IN. A rectangle of numbers written like

where each aij IR is called a matrix with m rows and n columns or an

Dana Mackey (DIT) Numerical Methods 1 / 56

A matrix composed of only one column (that is an m 1 matrix) is called

Dana Mackey (DIT) Numerical Methods 2 / 56

If A = [aij ] and B = [bij ] are two matrices in Mm,n then we define

Dana Mackey (DIT) Numerical Methods 3 / 56

If A = [aij ] Mm,n and k IR then we define the product of A with the

This operation is called scalar multiplication. Note that

Dana Mackey (DIT) Numerical Methods 4 / 56

Let n IN and let A be a 1 n matrix (a row matrix) and let Bbe an

Dana Mackey (DIT) Numerical Methods 5 / 56

Dana Mackey (DIT) Numerical Methods 6 / 56

Let m, n and p IN. Suppose A Mm,n and B Mn.p . Then we define

Dana Mackey (DIT) Numerical Methods 7 / 56

In particular we note that AB 6= BA in this case. Even if we take two

Dana Mackey (DIT) Numerical Methods 8 / 56

1 Interchange rows, e.g.

Any matrix can be converted to echelon form by using elementary row

Dana Mackey (DIT) Numerical Methods 10 / 56

Dana Mackey (DIT) Numerical Methods 11 / 56

Dana Mackey (DIT) Numerical Methods 12 / 56

A linear equation in one variable is an equation of the form

where a and b are given real numbers, and x is a variable.

A linear equation in n variables is an equation of the form

where a1 , . . . , an , b IR. For example, 2x1 + 3x2 = 7 is a linear equation in

Dana Mackey (DIT) Numerical Methods 13 / 56

a11 x1 + a12 x2 + + a1n xn = b1

The system of m equations above can also be written as one matrix

Suppose the system is AX = B where A Mm,n and B Mm,1 . Form the

Example: Solve the system of linear equations,

Dana Mackey (DIT) Numerical Methods 15 / 56

Solve, by Gaussian elimination, the following systems of linear equations.

Dana Mackey (DIT) Numerical Methods 16 / 56

Example: Consider the following system

Dana Mackey (DIT) Numerical Methods 17 / 56

0.6667x1 + 0.2857x2 + 0.2000x3 = 2.867

1 To obtain the 4-digit floating point approximation of a number, we

0.6667x1 + 0.2857x2 +0.2000x3 = 2.867

where the row operations performed were

0.1429 0.2857 0.4999 = 0.1429 0.1428 = 0.0001

0.6667x1 + 0.2857x2 +0.2000x3 = 2.867

which gives the solution:

Dana Mackey (DIT) Numerical Methods 20 / 56

Dana Mackey (DIT) Numerical Methods 21 / 56

For each column j let

Exercise: Apply the technique of Gaussian elimination with partial

Dana Mackey (DIT) Numerical Methods 22 / 56

Partial pivoting gives the solution x3 = 1.001, x2 = 7.000 and x1 = 1.000.

Dana Mackey (DIT) Numerical Methods 23 / 56

1 Show, by direct substitution, that the exact solution is (1,-1,1,-1).

Dana Mackey (DIT) Numerical Methods 24 / 56

Consider now the example

0.7x1 + 1725x2 = 1739

The exact solution is x1 = 20 and x2 = 1. Partial pivoting would leave the

0.7x1 + 1725x2 = 1739

hence x2 = 1.001 and x1 = 17.14.

Dana Mackey (DIT) Numerical Methods 25 / 56

If, instead, we choose the pivot which is largest in magnitude relative to

0.4352x1 5.433x2 = 3.271

which gives x2 = 1.000 and x2 = 20.000.

Dana Mackey (DIT) Numerical Methods 26 / 56

Si = max |aij |, for each 1i n