The Islamic University of Gaza

Faculty of Engineering
Civil Engineering Department

Numerical Analysis
ECIV 3306

Chapter 10
LU Decomposition and Matrix Inversion

Introduction
Gauss elimination solves [A] {x} ={B}
It becomes insufficient when solving these
equations for different values of {B}
LU decomposition works on the matrix [A] and the
vector {B} separately.
LU decomposition is very useful when the vector of
variables {x} is estimated for different parameter
vectors {B} since the forward elimination process is
not performed on {B}.

LU Decomposition
If:
L: lower triangular matrix
U: upper triangular matrix
Then,
[A]{X}={B} can be decomposed into two
matrices [L] and [U] such that:
1. [L][U] = [A] ([L][U]){X} = {B}

LU Decomposition
([L][U]){X} = {B}
Consider:
[U]{X} = {D}
So, [L]{D} = {B}
2. [L]{D} = {B} is used to generate an
intermediate vector {D} by forward substitution
3. Then, [U]{X}={D} is used to get {X} by back
substitution

Summary of LU Decomposition

LU Decomposition
As in Gauss elimination, LU decomposition must
employ pivoting to avoid division by zero and to

minimize round off errors. The pivoting is done

immediately after computing each column.

LU Decomposition
System of linear equations [A]{x}={B}

a11 a12
a
a
21
22

a31 a32

a13 x1 b1
l a21

a23 x2 b2 21
a11

a33 x3 b3

Step 1: Forward elimination

a11 a12 a13

[U ] 0 a22/ a23/
//
0
0 a33

a31
l31
a11

[ L][U ] [ A]

1 0

[L ] l 21 1
l 31 l 32

0
1

l32

a '32
a '22

LU Decomposition
Step 2: Generate an intermediate vector {D} by forward
substitution

1 0
l
21 1
l31 l32

0 d1 b1

0 d 2 b2
1 d 3 b3

[L]{D} = {B}

Step 3: Get {X} by back substitution.

a11 a12
0 a'
22

0
0

a13 x1 d1

a '23 x2 d 2
a' '33 x3 d 3

[U]{X}={D}

LU Decomposition-Example
3 0.1 0.2
A 0.1 7 0.3
0.3 0.2 10

0.2
3 0.1
0 7.003 0.293

0 0.19 10.02

0.1
0.3
l 21
0.03333; l 31
0.1000
3
3
a32\
0.19
l 32 \
0.02713
a22 7.003

0.2
3 0.1

U 0 7.003 0.293
0

0
10
.
012

0
0
1
[L ] 0.03333
1
0
0.1000 .02713 1

LU Decomposition-Example (contd)
Use previous [L] and {D} matrices to solve the system:
3 0.1 0.2 x1 7.85

0.1

7
0.31 x2 19.3

0.3 0.2 10 x3 71.4

Step 2: Find the intermediate vector {D} by forward substitution

0
0 d1 7.85 d1 7.85
1
0.0333
d 19.3 d 19.5617
1
0
2

2
0.1000 0.02713 1 d 3 71.4 d 3 70.0843

LU Decomposition-Example (contd)
Step 3: Get {X} by back substitution.

0.2 x1 7.85 x1 3
3 0.1
0 7.0033 0.2933 x 19.5617 x 2.5
2

2
0
0
10.012 x3 70.0843 x3 7.00003

Decomposition Step
% Decomposition Step
for k=1:n-1
[a,o]= pivot(a,o,k,n);
for i = k+1:n
a(i,k) = a(i,k)/a(k,k);
a(i,k+1:n)= a(i,k+1:n)-a(i,k)*a(k,k+1:n);
end
end

Partial Pivoting
%Partial Pivoting
function [a,o] = pivot(a,o,k,n)
[big piv]=max(abs(a(k:n,k)));
piv=piv+(k-1);
if piv ~= k
temp = a(piv,:);
a(piv,:)= a(k,:);
a(k,:)=temp;
temp = o(piv);
o(piv)=o(k);
o(k)=temp;
end

Substitution Steps
%Forward Substitution
d(1)=bn(1);
for i=2:n
d(i)=bn(i)-a(i,1:i-1)*(d(1:i-1))';
end
% Back Substitution
x(n)=d(n)/a(n,n);
for i=n-1:-1:1
x(i)=(d(i)-a(i,i+1:n)*x(i+1:n)')/a(i,i);
end

Matrix Inverse Using the LU

Decomposition
LU decomposition can be used to obtain the
inverse of the original coefficient matrix [A].

Each column j of the inverse is determined by

using a unit vector (with 1 in the jth raw).

Matrix Inverse: LU Decomposition

[A] [A]-1 = [A]-1[A] = I

Ax1 {b}1

Ax2 {b}2

Ax3 {b}3

1

0
0

0

1
0

0

0
1

Ax1
1st column
of [A]-1

Ax2

2nd column
of [A]-1

{x}1 {x}2 {x}3

Ax3

3rd column
of [A]-1

Matrix inverse using LU decomposition Example

3 0.1 0.2
A 0.1 7 0.3
0.3 0.2 10

0
0
1
[L ] 0.03333
1
0
0.1000 .02713 1

0.2
3 0.1
U 0 7.003 0.293
0
0
10.012

1A. [L]{d}1 = {b}1

0
0 d1 1 d1
1
1

0.0333
d 0 d 0.03333
1
0

2 2
0.1000 0.02713 1 d 3 0 d 3 0.1009 1st column

1B. Then, [U]{X}1={d}1

of [A]-1

0.2 x1 1
3 0.1
x1 0.33249
0 7.0033 0.2933 x 0.03333 x 0.00518
2

2
0
0
10.012 x3 0.1009 x3 0.01008

Matrix inverse using LU decomposition

Example (contd)
2A. [L]{d}2 = {b}2

0
0 d1 0 d1 0
1

0.0333

1
0 d 2 1 d 2 1

0.1000 0.02713 1 d 3 0 d 3 0.02713

2B. Then, [U]{X}2={d}2

2nd column
of [A]-1

0.2 x1 0 x1 0.004944
3 0.1
0 7.0033 0.2933 x 1 x 0.142903
2

2
0
0
10.012 x3 0.02713 x3 0.00271

Matrix inverse using LU decomposition

Example (contd)
3A. [L]{d}3 = {b}3
0
0 d1 0 d1 0
1
0.0333
d 0 d 0
1
0

2 2
0.1000 0.02713 1 d 3 1 d 3 1

3B. Then, [U]{X}3={d}3

3rd column
of [A]-1

0.2 x1 0 x1 0.006798
3 0.1
0 7.0033 0.2933 x 0 x 0.004183

2 2
0
0
10.012 x3 1 x3 0.09988

Matrix inverse using LU decomposition

Example (contd)

0.33249 0.004944 0.006798

[ A]1 0.00518 0.142903 0.004183
0.01008 0.00271 0.09988

ERROR ANALYSIS AND SYSTEM CONDITION

The inverse provides a means to discern whether
systems are ill-conditioned.
1. Scale the matrix of coefficients [A] so that the largest
element in each row is 1. Invert the scaled matrix and if
there are elements of [A]1 that are several orders of
magnitude greater than one, it is likely that the system is illconditioned.
2. Multiply the inverse by the original coefficient matrix and
assess whether the result is close to the identity matrix. If
not, it indicates ill-conditioning.
3. Invert the inverted matrix and assess whether the result is
sufficiently close to the original coefficient matrix. If not, it
again indicates that the system is ill-conditioned.

Vector and Matrix Norms

Norm is a real-valued function that provides a measure of
size or length of vectors and matrices.
Norms are useful in studying the error behavior of
algorithms.

A simple example is a vector in three - dimensiona l Euclidean space

that can be represented as
F a b c
where a, b, and c are the distances along x, y, and z axes, repectively.

Vector and Matrix Norms (contd)

Vector and Matrix Norms (contd)

The length of this vector can be simply computed as

F e a 2 b2 c2
Length or Euclidean norm of [F]

For an n dimensional vector

X x1 x2 xn
a Euclidean norm is computed as
Xe

2
x
i
i 1

For a matrix [A]

Ae

Frobenius norm

i 1 j 1

ai2, j

(10.24)

Vector and Matrix Norms (contd)

For vectors, there are alternatives called p norms
that can be represented generally by

P-Norm
1/ p

X p xi
i 1

1-Norm
We can also see that Euclidean norm and the 2
norm, ||X||, For
areVector
identical for vectors.
n

X 1 xi

i 1
In contrast to vectors,
the 2 norm and Euclidean
For Matrix
(Column
Sum Norm)
norm for amatrix
are not
the same.

1/ p

X p xi
Vector
and
Matrix
i 1

1-Norm
For Vector
n

Norms (contd)

X 1 xi
i 1

For Matrix (Column Sum Norm)

n

A 1 max ai , j
1 j n

i 1

Uniform vector norm X max x

Uniform matrix norm (row sum Norm)

max ai , j
1i n

j 1

1i n

Vector and Matrix Norms (contd)

Matrix Condition Number Defined as:

Cond A A A

For a matrix [A], this number will be greater than or

equal to 1.
Relative error of the norm X

Cond A

A
A

For example, if the coefficients of [A] are known to

t-digit precision (rounding errors~10-t) and
Cond [A]=10c, the solution [X] may be valid to only
t-c digits (rounding errors~10c-t).

The condition number is

considerably greater than
unity suggests that the
system is ill-conditioned.

Iterative Refinement
Round-off errors can be reduced by the following procedure:

a11x1 a12 x2 a13 x3 b1

a21x1 a22 x2 a23 x3 b2
a31x1 a32 x2 a33 x3 b3

Suppose an approximate solution vectors given by

~ T ~ ~ ~
X [ x1 x2 x3 ]

Substitute the result in the original system

~
~
~
~
a11x1 a12 x2 a13 x3 b1
~ ....(Eq.1)
~
~
~
a21x1 a22 x2 a23 x3 b2
~
~
~
~
a31x1 a32 x2 a33 x3 b3

Iterative Refinement (contd)

Now, assume the exact solution is:
x1 ~
x1 x1
x ~
x x
2

x3 ~
x3 x3

Then:
a11( ~
x1 x1 ) a12 ( ~
x2 x2 ) a13 ( ~
x3 x3 ) b1
a21( ~
x1 x1 ) a22 ( ~
x2 x2 ) a23 ( ~
x3 x3 ) b2 .(Eq.2)
a (~
x x ) a ( ~
x x ) a ( ~
x x ) b
31

32

33

Iterative Refinement (contd)

Subtract Eq.2 from Eq.1, the result is a system of linear
equations that can be solved to obtain the correction factors x

~
a11x1 a12x2 a13x3 b1 b1 E1
~
a21x1 a22x2 a23x3 b2 b2 E2
~
a31x1 a32x2 a33x3 b3 b3 E3
The factors then can be applied to improve the solution as

~
x

specified by the equation: 1 x1 x1

x ~
x x
2

x3 ~
x3 x3

Iterative Refinement- Example (contd)

Solve: 4.23x1 1.06 x2 2.11x3 5.28

2.53x1 6.77 x2 0.98 x3 5.22

1.85 x1 2.11x2 2.32 x3 2.58
The exact solution is

X T [1.00

1.00 1.00]

1- Solve the equations using [A]-1, such as {x}=[A]-1{c}

~ T
X [0.991 0.997 1.00]

Iterative Refinement - Example

(contd)
2- Substitute the result in the original system [A]{x}={c}

~
~
[ A]{X } {C}

5.24511
~
{C} 5.22246

2.59032

5.28 5.24511 0.0348

~
E {C} {C} 5.22 5.22246 0.00246
2.58 2.59032 0.0103

Iterative Refinement - Example

(contd)
~
3- Solve the system of linear equations [ A]{X } {E}

using [A]-1 to find the correction factors x to yield

X T [0.00822

0.00300 0.00000757]

x1 ~
x1 x1 0.999
x ~
x x 1.00
2

x3 ~
x3 x3 1.00