Lecture 4: SYSTEM OF LINEAR

ALGEBRAIC EQUATIONS

Dr. Nor Alafiza Yunus

Faculty of Chemical and Energy Engineering
Universiti Teknologi Malaysia, 81310 UTM
Johor Bahru, Malaysia
1
Students will learn how to:
 Solve problems involving linear algebraic
equations
 Appreciate the usage of linear algebraic
equations in any field of engineering
 Mastering several techniques and their
reliability
 Naïve Gauss elimination
 Gauss-Siedel method
 Apply MATLAB program to successfully solve
systems of linear algebraic equations
2
 Introduction

 Naïve Gauss Elimination

 Gauss-Siedel Method
 Solving linear equations using MATLAB

3
 Many of the fundamental equations of
engineering and science are based on
conservation laws.
 Some familiar quantities that conform to such
laws are mass, energy, and momentum.
 In mathematical terms, these principles lead
to balance or continuity equations that relate
system behavior as represented by the levels
or response of the quantity being modeled

4
Two types of systems that can be modeled using linear algebraic equations:
(a) lumped variable system that involves coupled finite components and
(b) distributed variable system that involves a continuum.

5
 In lecture 2, the value x that satisfied a single
equation, f(x) = 0 is determined.
 Now, we deal with the case of determining the
values x1, x2, …… xn that simultaneously satisfy a set
of equations:
f1 (x1, x2, . . . xn ) = 0
f2 (x1, x2, . . . xn ) = 0
. .
. .
fn (x1, x2, . . . xn ) = 0

6
For system of linear algebraic equation in general
form:
a11x1 + a12x2 + . . . .+ a1nxn = b1
a21x1 + a22x2 + . . . .+ a2nxn = b2
. . .
. . .
an1x1 + an2x2 + . . . .+ annxn = bn ------- (3.1)
where;
a = constant coefficients
b = constants
n = number of equations
Note: Only linear equations involve in the calculation.
7
 There are 4 approaches for solving a
system of linear algebraic equations:
 Naïve Gauss Elimination (Elimination
method)
 Gauss-Siedel Method (iterative solution
techniques)
 Gauss-Jordan Elimination
 LU Decomposition Method

8
 Introduction
 Naïve Gauss Elimination

 Gauss-Siedel Method
 Solving linear equations using MATLAB

9
 There are 2 steps of procedure
for solving:
i) Step 1-Forward elimination:
The equations were
manipulated to eliminate
one of unknowns from the
equations. The result - one
equation with one unknown.

ii) Step 2-Back substitution: This

equation could be solved
directly & the result back-
substituted into one of the
original equations to solve
for remaining unknowns.

10
To solve a general set of n equations (3.1):
a11x1 + a12x2 + . . . .+ a1nxn = b1
a21x1 + a22x2 + . . . .+ a2nxn = b2
. .
. .
an1x1 + an2x2 + . . . .+ annxn = bn------- (3.1)

where; a = constant coefficients

b = constants
n = number of equations

11
In simplified form;
Ax = b -------- (3.2)
where; A = known non-singular matrix n x n
b = n x 1 matrix
x = unknown nx1 matrix (to be
calculated)

x and b also called as vector column / vectors

To solve equation (3.1) or (3.2) we use direct
method of forward elimination and back
substitution without iterative.
12
Example 3.1:
Solve for vector x1, x2, and x3; as describe by linear
equations:
3x1 + 2x2 – x3 = 5 -------- (R1)
x1 - x2 + 3x3 = -10 ------ (R2)
-2x1 + x2 – 2x3 = 5 ------ (R3)
Transform into matrix

3 2 −1 𝑥1 5
1 −1 3 𝑥2 = −10
−2 1 −2 𝑥3 5 13
1. Simplified matrix
3 2 −1 ⋮ 5 (𝑅1) < - - pivotal row
1 0 −1 3 ⋮ −10 (𝑅2)
−2 0 1 −2 ⋮ 5 (𝑅3)

2. Eliminates x1 from R2 & R3 by multiply R1 with

coefficient.
Now, R1 is called as pivotal row.
Pivot element, a11 = 3
Coefficient, m1 = - a21/a11 = -1/3
m2 = - a31/a11 = -(-2/3) = 2/3

14
 3. Then, eliminate a21: R2’ = R2 + m1R1
eliminate a31: R3’ = R3 + m2R1
m1 = -1/3 m2 =2/3

R2’: a21=1-1/3(3)=0 a22=-1-1/3(2)=-5/3 a23=3-1/3(-1)=10/3 b21=-10-1/3(5)=-35/3

R3’: a31=-2+2/3(3)=0 a22=1+2/3(2)=7/3 a23=-2+2/3(-1)=-8/3 b21=5+2/3 = 25/3

3 2 −1 5
3 2 −1 ⋮ 5 R2'=R2− 1/3 R1 5 10 ⋮ 35 𝑅1
1 −1 3 ⋮ −10 0 − ⋮ − 𝑅2′
3 3 3
R3’=R3+ 2/3 R1 7 8 ⋮ 25
−2 1 −2 ⋮ 5 0 − 𝑅3′
3 3 3

15
4. Eliminates x2 from R3’ by using R2’ as pivotal row
Pivot element, a22 = -5/3
Coefficient, m3 = - a32/a22 = -(7/3)/(-5/3) = 7/5

7 7 5
R3’’ 𝑎32 = + − =0
3 5 3
8 7 10
𝑎33 = − + =2
3 5 3
25 7 35
𝑏31 = + − = −8
3 5 3

3 2 −1 5 3 2 −1 ⋮ 5
5 10 ⋮ 35 R3''=R3' +7/5 R2' 𝑅1
0 − ⋮ − 0 −
5 10
⋮ −
35
𝑅2′
3 3 3
7 8 3 3 3
0 − ⋮ 25 0 0 2 ⋮ −8 𝑅3′′
3 3 3

16
5: Calculate x1, x2, and x3 by back substitution.
x3 = b3 /a33 = -8/2 = -4
x2 = (b2 – a23x3 )/a22 = [-35/3 – 10/3(-4) ]/(-5/3) = -1
x1 = (b1 – a12x2 – a13x3)/a11 = [5 – 2(-1) – 4 ]/3 =1

Therefore: x1=1, x2= -1, x3 = -4

17
 Answer: x1 1
x= x2 = -1
x3 -4
or
x = [1, -1, 4]T
where T = transpose matrix

18
1. In every steps, pivot element  0
2. If pivot element = 0, rearrange
the rows to lower level until pivot
element  0
3. If pivot element = 0 after rearrange;
no single solution or answer for vector x.

19
Given a set of linear equations:
10𝑥1 + 2𝑥2 − 𝑥3 = 27
− 3𝑥1 − 5𝑥2 + 2𝑥3 = −61.5
𝑥1 + 𝑥2 + 6𝑥3 = −21.5

a) Solve by naive Gauss elimination. Show all

steps of the computation.
b) Substitute your results into the original
equations to check your answers.
ANSWER: x1 = 0.1525
x2 = 10.0922
x3 = -5.2908 20
1. An electrical engineer supervises the production of three types of
electrical components. Three kinds of material - metal, plastic, and
rubber - are required for production. The amounts needed to produce
each component are given in Table. If totals of 2.12, 0.0434, and 0.164
kg of metal, plastic, and rubber, respectively, are available each day,
how many components can be produced per day? Use 4 decimal place
throughout the calculation.
1. Solve the problem using Naïve Gauss Elimination method
2. Write the MATLAB commands to solve the problem using
NaiveGauss user-define function MATLAB

Component Metal (g/ Plastic Rubber

component) (g/component) (g/component)
1 15 0.25 1.0
2 17 0.33 1.2
3 19 0.42 1.6
21
Use naïve Gauss Elimination method to solve

2𝑥2 + 3𝑥3 = 8
4𝑥1 + 6𝑥2 + 7𝑥3 = −3
2𝑥1 + 𝑥2 + 6𝑥3 = 5

22
 Division by zero
During both the elimination and back substitution
phase, it is possible that a division by zero could
occur
2𝑥2 + 3𝑥3 = 8
4𝑥1 + 6𝑥2 + 7𝑥3 = −3 a11 = 0
2𝑥1 + 𝑥2 + 6𝑥3 = 5

 Round-Off Errors
Since computers carry a limited number of
significant figures, round off errors can occur and it
will propagate.
23
 ILL Conditioned systems
Adequacy of solution depends on condition of
the system. ILL conditioned system will
produced a wide range of answers.

24
 Using “Partial Pivoting”
 Example: Division by zero
 Determine the largest available coefficient in the
column below the pivot element.
 Switched the row so that the largest element is the
pivot element. pivot element

2𝑥2 + 3𝑥3 = 8 4𝑥1 + 6𝑥2 + 7𝑥3 = −3

4𝑥1 + 6𝑥2 + 7𝑥3 = −3 2𝑥2 + 3𝑥3 = 8
2𝑥1 + 𝑥2 + 6𝑥3 = 5 2𝑥1 + 𝑥2 + 6𝑥3 = 5

25
 Example: Round off error

1) Without switched the row

0.0003x1 + 3.0000x2 = 2.0001 .......(i)

1.0000x1 + 1.0000x2 = 1.0000 .......(ii)

Pivot element, a11=0.0003 is very close to zero.

26
Solve by using Gauss elimination:
0.0003x1 + 3.0000x2 = 2.0001 ....... (i)
1.0000x1 + 1.0000x2 = 1.0000 ....... (ii)

Step 1: eliminates x1 from equations (ii)

Pivot element, a11 = 0.0003
Coefficient, m1 = -a21/a11 = -(1.0000/0.0003)

Then, eliminate a21: equation (ii) + (m1)(i)

(ii) : 1.0000x1 + 1.0000x2 = 1.0000
(m1)(i) : -1.0000x1 - 10,000x2 = -6667
 2 
-9999x2 = -6666 2.0001 3  
 3  1
 x2 = 2/3 x1  
0.0003 3

Substituted x2 into Eq.(i):

27
 However the result is very sensitive to the number
of significant figures carried in the computation.

Significant x2 x1 Absolute value of

Figures percent relative
error for x1
3 0.667 -3.33 1099

4 0.6667 0.0000 100

5 0.66667 0.30000 10

6 0.666667 0.330000 1

7 0.6666667 0.3330000 0.1

28
 If equations are solved in reverse order, switched row with
the largest pivot element:

0.0003x1 + 3.0000x2 = 2.0001 .......(i)

1.0000x1 + 1.0000x2 = 1.0000 .......(ii)

 Rearrange the equations

1.0000x1 + 1.0000x2 = 1.0000 ...(1)
0.0003x1 + 3.0000x2 = 2.0001 ...(2)

Elimination and substitution yield:

x2 = 2/3
x1 = (1-(2/3))/1 = 1/3

29
 This case is much less sensitive to the number significant
figures in the computation

Significant x2 x1 Absolute value of

Figures percent relative
error for x1
3 0.667 0.333 0.1

4 0.6667 0.3333 0.01

5 0.66667 0.33333 0.001

6 0.666667 0.333333 0.0001

7 0.6666667 0.3333333 0.00001

30
 Introduction
 Naïve Gauss Elimination
 Gauss-Siedel Method

 Solving linear equations using MATLAB

31
To solve a general set of n equations (3.1):
a11x1 + a12x2 + . . . .+ a1nxn = b1
a21x1 + a22x2 + . . . .+ a2nxn = b2
. .
. .
an1x1 + an2x2 + . . . .+ annxn = bn------- (3.1)

where; a = constant coefficients

b = constants
n = number of equations
33
 Previously, linear equations is solved using elimination
method.
 Now, we use iterative method. The most commonly used
iterative method is Gauss-Siedel method.
 Note:
 For conciseness, this method is limited to 3 x 3 set of equations only
 If the diagonal elements are nonzero, the first equation can be solved
for x1, the second for x2, and the third for x3 to yield:
10𝑥1 + 2𝑥2 − 𝑥3 = 27 … … … (1) (1)  x1
− 3𝑥1 − 5𝑥2 + 2𝑥3 = −62 … (2) SOLVE (2)  x2
𝑥1 + 𝑥2 + 6𝑥3 = −22 … … … (3)
(3)  x3
10 2 −1
−1 −5 2 Diagonal element
1 1 6 34
 j 1 j 1
b1  a12 x2  a13 x3 ... 12.1(a)
x1 
j

a11
j 1
b2  a21 x1  a23 x3
j
x2  ... 12.1(b)
j

a22
b3  a31 x1  a32 x2
j j
x3 
j ... 12.1(c)
a33
xij  xij 1
 a ,i  j
100%   s ... 12.2
xi
35
1. Choosing guesses values for x2 j-1 and x3 j-1. A simple way to
obtain initial guesses is to assume that they are all zero.
2. Calculate a new value for x1=b1/a11 by substitute these
initial guess into Eq. 12.1 (a)
3. Compute a new value for x2 by substitute new value of x1
along with the previous initial value of x3 in Eq. 12.1 (b)
4. Calculate a new estimate for x3 by substitute x1 and x2
obtained from step 2 and 3.
5. Check convergence using Eq. 12.2 for all i, where j and (j-1)
are the present (new) and previous iterations(old).
6. Then, return to the first equation and repeat the entire
procedure until all our solution converges closely enough to
the true values.
37
Find X1, X2 and X3 using Gauss-Siedel method with εs=0.05%
10𝑥1 + 2𝑥2 − 𝑥3 = 27 … … … (1)
− 3𝑥1 − 5𝑥2 + 2𝑥3 = −62 … (2)
𝑥1 + 𝑥2 + 6𝑥3 = −22 … … … (3)
1) Let x2old=0, x3old=0
2) Calculate x1
27 − 2𝑥2 + 𝑥3
𝑥1 𝑛𝑒𝑤 =
10
27 − 2(0) + (0)
𝑥1 𝑛𝑒𝑤 = = 2.700
10
3) Calculate x2, x1=2.7, x3=0;

𝑛𝑒𝑤
−62 + 3𝑥1 − 2𝑥3
𝑥2 =
−5
−62 + 3(2.7) − 2(0)
𝑥2 𝑛𝑒𝑤 = = 10.78 38
4) Calculate x3, x1=2.7, x2=10.78;
−22 − 𝑥1 − 𝑥2
𝑥3 𝑛𝑒𝑤 =
6

𝑛𝑒𝑤
−22 − 2.7 − (10.78)
𝑥3 = = −5.9133
6
5) Calculate εa,1, εa,2 ,and εa,3
𝜀𝑎,1 = −
10.78−0
𝜀𝑎,2 = × 100 = 100%
10.78
−5.9133−0
𝜀𝑎,3 = × 100 = 100%
−5.9133

6) Repeat Step 2 to calc. x1, using x2= 10.78, x3= - 5.9133

27 − 2(10.78) + (−5.9133)
𝑥1 𝑛𝑒𝑤 = = −0.0473
10

39
3) Calculate x2, x1=−0.0473, x3= - 5.9133 ;

𝑛𝑒𝑤
−62 + 3(−0.0473) − 2(− 5.9133)
𝑥2 = = 10.0631
−5
4) Calculate x3, x1= −0.0473, x2=10.0631;

𝑛𝑒𝑤
−22 + 0.04732.7 − (10.0631)
𝑥3 = = − 5.3360
6
5) Calculate εa,1, εa,2 ,and εa,3

−0.0473−2.7
𝜀𝑎,1 = × 100 = 5804.22%
−0.0473
10.0631−10.78
𝜀𝑎,2 = × 100 = 0.3802%
10.0631
−5.3360−(−5.9133)
𝜀𝑎,3 = × 100 = 17.1053%
−5.3360

40
X1=0.1277 X2=10.1702 X3= -5.3830

41
 Solve for X, Y and Z using Gauss-Siedel
method, es= 5%
31.3𝑋 − 11.2𝑌 − 5.8𝑍 = 21.54 … … … (1)
− 12.7𝑋 + 45.4𝑌 + 8.06𝑍 = 17.43 … (2)
4.65𝑋 + 𝑌 + 10.04𝑍 = −12.47 … … … (3)

Answer: 4 iterations
X= 0.6911 Y= 0.8698 Z= -1.6487
42
Solve for vector x1, x2, and x3; as describe by linear equations
using Gauss-Siedel method. Stop iteration at |es|=5%:
3x1 + 2x2 – x3 = 5 -------- (i)
x1 - x2 + 3x3 = 10 ------- (ii)
-2x1 + x2 – 2x3 = 5 ------ (iii)

43
ANSWER for Exercise 3.4 ???

How many iterations?

44
DIVERGE

45
 Gauss-Seidel will converge if the following condition
holds: 𝑛

𝑎𝑖𝑖 > ෍ 𝑎𝑖𝑗

𝑗=1
𝑗≠𝑖
 That is, the absolute value of the diagonal coefficient in
each of the equations must be larger than the sum of
the absolute values of the other coefficients in the
equation.
 This criterion is sufficient but not necessary for
convergence.
 the method may sometimes work if Eq above is not met,
convergence is guaranteed if the condition is satisfied.

47
 𝑛

3 2 −1 𝑎𝑖𝑖 > ෍ 𝑎𝑖𝑗

1 −1 3 𝑗=1
𝑗≠𝑖
−2 1 −2

i=1, 3 > 2+1 3>3?

i=2, 1 > 1+3 1>4? DIVERGE
i=3, 2 > 2+1 2>3?

48
0. Check the convergence and diagonal dominance
1. Choosing guesses values for x2 j-1 and x3 j-1. A simple way to
obtain initial guesses is to assume that they are all zero.
2. Calculate a new value for x1=b1/a11 by substitute these
initial guess into Eq. 12.1 (a)
3. Compute a new value for x2 by substitute new value of x1
along with the previous initial value of x3 in Eq. 12.1 (b)
4. Calculate a new estimate for x3 by substitute x1 and x2
obtained from step 2 and 3.
5. Check convergence using Eq. 12.2 for all i, where j and (j-1)
are the present (new) and previous iterations(old).
6. Then, return to the first equation and repeat the entire
procedure until all our solution converges closely enough to
the true values. 49
Solve for vector x1, x2, and x3; as describe by linear equations
using Gauss-Siedel method. Stop iteration at |es|=5 %:
3x1 + 2x2 – x3 = 5 -------- (i)
x1 - x2 + 3x3 = 10 ------- (ii)
-2x1 + x2 – 2x3 = 5 ------ (iii)

50
Solve for vector x, y, and z; as describe by linear equations using
Gauss-Siedel method. Stop iteration at |es|%=5:
8x + 3y +z = 12 -------- (i)
- 6x + 7z = 1 ------- (ii)
2x + 4y – z = 5 ------ (iii)

51
2. The equations given below can be solved to find the standard ratio
of macronutrients in the organic fertilizers that a company should
manufacture per day. The unknowns are the number of nitrogen (N),
phosphorus (P), and potassium (K).
3𝑁 + 7𝑃 + 13𝐾 = 76 … … … (1)
𝑁 + 5𝑃 + 3𝐾 = 28 … … … (2)
12𝑁 + 3𝑃 − 5𝐾 = 1 … … … (3)
 Find the values of N,P,K and absolute relative error for each
nutrient using N=1, P=0, and K=1 as initial guess. Conduct iteration
until |εa,i|% < 5
 Solve the linear algebraic problem using simple 3-lines MATLAB
command.
 Write the MATLAB commands to solve the problem using
GaussSeidel user-define function (given in text book page 289)

54
 Introduction
 Naïve Gauss Elimination
 Gauss-Siedel Method
 Linear algebraic equations and MATLAB

57
 General form of linear algebraic
[A] {x}={b}
{x}=[A]-1{b}

1) Simple MATLAB command

a) Using backslash or “left-division”
x= A\b

b) Using matrix inversion

x=inv(A)*b
65
2) Using GaussNaive user-defined function

66
3) Using GaussSeidel
user-defined function

67
1) Solve for vector x1, x2, and x3; as describe by
linear equations:
3x1 + 2x2 – x3 = 5 -------- (i)
x1 - x2 + 3x3 = 10 ------- (ii)
-2x1 + x2 – 2x3 = 5 ------ (iii)
Solve using all methods

68
% Simple method % Gauss Naive function
A=[ 7 2 -3; 2 5 -3;1 -1 -6]; A=[ 7 2 -3; 2 5 -3;1 -1 -6];
b=[-12; -20;-26]; b=[-12; -20;-26];
Y=GaussNaive(A,b)
W=A\b
X=inv(A)*b

% Gauss Seidel function

A=[ 7 2 -3; 2 5 -3;1 -1 -6];
b=[-12; -20;-26];
Z=GaussSeidel(A,b)

69
2) 2x1 − 0.1x2 − 0.2x3 = 7
0.1x1 + 5x2 − 0.3x3 = −20
0.3x1 − 0.2x2 + 9x3 = 70

70
3. A company is planning to invest a total of RM 1 million into 3 projects, A, B
and C. The estimated one-year return for each project under two different
market conditions, I and II is given on table. The manager’s goal is to
distribute funds between the projects in a way that would yield the total
return of 1.2 for both scenarios. The amount of money (in millions of ringgit)
to be invested in each project are denote as XA, XB, XC, respectively.
Project Scenario I Scenario II
A 1.4 0.9
B 0.8 2.0
C 1.2 1.0
Total 1.2 1.2

a) Formulate the problem as mathematical equation

b) Write (a) in the matrix form
c) Write simple three lines MATLAB commands to solve for XA, XB, and XC
d) Solve the problem using Naïve Gauss elimination method

71
Following is the matrix operation:
{x} = {d} − [C]{x}
Where
𝑏1 𝑎12 𝑎13
ൗ𝑎11 0 Τ𝑎11 Τ𝑎11
𝑎23
{d} = 𝑏2
ൗ𝑎22 [C] = 𝑎21Τ𝑎22 0 Τ𝑎22
𝑏1 𝑎31 𝑎32
ൗ𝑎33 Τ𝑎32 Τ𝑎33 0

for i = 1:n while (1)

C(i,1:n)= ………………………; xold = x;
end for i = 1:n
x(i) = …………………………...........…;
for i = 1:n if x(i) ~= 0
d(i)= ………………………………; ea(i) = abs((x(i) - xold(i))/x(i)) * 100;
end end
iter = 0; end
end
72