SYSTEM OF

LINEAR EQUATIONS
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Physical problems involved the interaction
of various properties
Systems of equations
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
m m mn n m
a x a x a x b
a x a x a x b
a x a x a x b
+ + + =
+ + + =
+ + + =
Equations are all linear System of linear equations
The system has only 3 solutions:
Unique
Infinite
No solution
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
SOLVING SYSTEM OF LINEAR EQUATIONS
Elimination method
LU factorization
method
Iteration method
Elimination method
Gauss
elimination
Gauss elimination
with pivoting
Only discuss UNIQUE solution where matrix A must be
square matrix (n x n)
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Gauss elimination
A set of linear equations (n x n system):
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
n n
n n
n n nn n n
a x a x a x b
a x a x a x b
a x a x a x b
+ + + =
+ + + =
+ + + =
11 1 1 1
1
n
n nn n n
a a x b
a a x b
| | | | | |
| | |
=
| | |
| | |
\ . \ . \ .
b
x
Matrix
form
nxn
A
Augmented
matrix form
11 1 1
1
n
m mn n
a a b
a a b
| |
|
|
|
\ .
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
11 12 13 1
22 23 2
33 3
0
0 0
u u u f
u u f
u f
| |
|
|
|
\ .
11 12 13 1
21 22 23 2
31 32 33 3
a a a b
a a a b
a a a b
| |
|
|
|
\ .
R1
Row
We need to
eliminate all
these values
The matrix becomes
An idea:
Diagonal
Upper triangular system
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Step of solving:
Given
11 12 13 1
21 22 23 2
31 32 33 3
a a a b
a a a b
a a a b
| |
|
|
|
\ .
Do row operation to eliminate values,
eliminate values in column order.
i =respective row, j =respective column
New
ij
j i i
jj
a
R R R
a
| |
+
|
\ .
The system will becomes upper triangular system.
11 12 13 1
22 23 2
33 3
0
0 0
u u u f
u u f
u f
| |
|
|
|
\ .
Solve using backward substitution.
Example: (Gauss elimination)
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Gauss elimination with pivoting
Solve the problem of division by zero (in case where the value
at diagonal is zero) and minimize the loss of precision caused
by accumulated round-off errors in numerical calculations.
The algorithm is almost the same with Gauss elimination method
except we have to
2. Do elimination process.
1. Swap rows (if necessary) to get the maximum .
Example: (Gauss elimination with pivoting)
11
| | a
3. Repeat step 1 (for maximum diagonal value) and 2 until
last column.
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
LU factorization method
Doolittle
Crout
Thomas
Thomas algorithm
Cholesky
------------------------------------------------------------------------------------------
b Ax =
Linear equations :
LU A =
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
11 12 1
22 2
0
0 0
n
n
nn
u u u
u u
u
| |
|
|
|
|
\ .
11
21 22
1 2
0 0
0
n n nn
l
l l
l l l
| |
|
|
|
|
\ .
Lower triangular matrix
Upper triangular matrix
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
21
1 2
1 0 0
1
0
1
n n
l
L
l l
| |
|
|
=
|
|
\ .
Doolittle method
LU A=
Diagonal matrix L = 1
L
1. , determine L and U
LU A =
2. , solve for y
b Ly =
3. , solve for x y Ux =
Example: (Doolittle)
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
12 1
2
1
0 1
0 0 1
n
n
u u
u
U
| |
|
|
=
|
|
\ .
Crout method
LU A=
Diagonal matrix U = 1
U
1. , determine L and U
LU A =
2. , solve for y
b Ly =
3. , solve for x y Ux =
Example: (Crout)
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
1 1
2 2 2
3
1
0 0
0 0
0 0
n
n n
d e
c d e
c
e
c d

| |
|
|
|
|
|
|
\ .
Thomas method
Condition Matrix A must be tridiagonal
1
2
3
If the matrix is not tridiagonal Rearrange
12 1
2
1
0 1
0 0 1
n
n
u u
u
U
| |
|
|
=
|
|
\ .
LU A=
Diagonal matrix U = 1
U
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Example: (Thomas )
Thomas algorithm
Generalization of Thomas method
Suitable to solve larger system
1. , determine L and U
LU A =
2. , solve for y
b Ly =
3. , solve for x y Ux =
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
1 1 1 1
2 2 2 2 2
3
1
0 0
0 0
0 0
n
n n n n
d e x b
c d e x b
c
e
c d x b

| || | | |
| | |
| | |
= | | |
| | |
| | |
| | |
\ .\ . \ .
Condition Matrix A must be tridiagonal
If the matrix is not tridiagonal Rearrange
i
c
i
d
i
e
i
b
Formula: Thomas algorithm
i
x
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Table
Example: (Thomas algorithm)
i
x
i
y
i
|
i
o
i
b
i
c
i
e
i
d
i
Values from linear system
Use formula of Thomas algorithm
1 2

n
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Cholesky method
Condition Matrix A must be symmetric positive definite
Definition Matrix A symmetric
positive definite if


(see example 3.13)
0 , 0 = > x Ax x
T
Properties
1. 0 = A
2.
n i a
ii
, , 2 , 1 , 0 = >
3.

ii kj
a a
n i
n j
n k
max max
1
1
1 s s
s s
s s
s
4.

2
( ) , , 1, 2, ,
ij ii jj
i j
a a a i j n

< =
"
K
A symmetric matrix A is
positive definite if and
only if each of its
leading principal
submatrix has a
positive determinant
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
11 21 1
22 2
0
0 0
n
n
nn
l l l
l l
U
l
| |
|
|
=
|
|
\ .
LU A=
U = L
T
U
1. , determine L and U
LU A =
2. , solve for y
b Ly =
3. , solve for x y Ux =
Example: (Cholesky)
DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi
Iteration method
Gauss-Seidel
Convergence criteria Matrix A must be diagonally dominant
If the matrix is not diagonally dominant Rearrange
Example: (Diagonally dominant)
.
1
, 1, 2,
n
ii ij
j
j i
a a i n
=
=
> =

DEPARTMENT OF MATHEMATICS AND STATISTICS, FSTPi

Table
Formula
1
( 1) ( )
1 1
( 1)
, 1, 2, 3,..,
i n
k k
i ij j ij j
j j i
k
i
ii
b a x a x
x i n
a

+
= = +
+

= =

k
( )
1
k
x
( )
2
k
x
( ) k
n
x
0
1
Stopping criteria
Initial guess
Use formula
Stopping criteria { }
1
( 1) ( )
| |
max
i n
k k
i i
x x c
s s
+
< 0.005 c =
Example: (Gauss-Seidel)