Linear Algebraic Systems I

1. Existence and uniqueness of solutions

2. Determinants and matrix inverses
3. Gauss-Jordan elimination
4. Ill-conditioned matrices
 Fundamental Theorem
 Linear algebraic system
 a11  a1n b1 
Ax  b  A  A b       
~

am1  amn bm 

 Consistency
~
» The system has solutions if and only if the matrices A and A
have the same rank r

 Uniqueness
» The system has a single solution if and only if both matrices have
rank r = n

 Infinitely many solutions

» The system has infinitely many solutions if and only if both
matrices have rank r < n
 Implications
 Homogeneous system
 a11  a1n 0
Ax  0  A  A b       
~

am1  amn 0

» Trivial solution: x = 0
» Nontrivial solutions exist if and only if rank(A) < n
» Nontrivial solutions are said to be contained in the null
space of A
 Nonhomogeneous system
» If the system is consistent, then all solutions can be
represented as x = x0+xh
» x0 is a particular solution of the nonhomogeneous system
» xh is any solution of the homogeneous system
 Fundamental Theorem Examples
 Unique solution
1 2 1 rank ( A)  2  1
3 5 x  2 rank ( A
~
)2
x 
    1
 Infinitely many solutions
1 2 1 rank ( A)  1 1  2a 
3 6 x  3 rank ( A
~

x 
    ) 1  a 

 No solutions
1 2 1 rank ( A)  1
3 6 x  1 rank ( A
~
)2
No solutions exist
  
 Determinants
 Only applicable to square matrices
 Notation: det(A), |A|
 2x2 matrix

a11 a12
 a11a22  a12a21
a21 a22

 3x3 matrix
a11 a12 a13
a11a22a33  a12a23a31  a13a21a32
a21 a22 a23 
 a11a23a32  a12a21a33  a13a22a31
a31 a32 a33
 More general formulas based on cofactors are presented
in the text
 Determinant Examples

1 2
 (1)( 4)  (2)(3)  2
3 4

1 2 3
(1)( 4)(9)  (2)(6)(7)  (3)( 4)(8)
4 5 6 0
 (1)(6)(8)  (2)( 4)(9)  (3)(5)(7)
7 8 9
 Properties of Determinants
 |A| = |AT|
 Diagonal & triangular matrices
a11 0  0 a11 a12  a1n a11 0  0
0 a22  0 0 a22  a1n a21 a22  0
   a11a22  ann
           
0 0  ann 0 0  ann an1 an 2  ann

 Products: |AB| = |A||B|

 A zero column or row produces a zero determinant
 Linearly dependent rows or columns produce a zero
determinant
 A square matrix A has full rank n if and only if |A| is
non-zero
 Matrix Inverse
 Definition
» Assume A is a nxn matrix
» The inverse of A is denoted A-1
» The inverse satisfies the equations: AA 1  A 1A  I
 Existence & uniqueness
» The inverse exists if and only if: det( A)  0
» If A has an inverse, then the inverse is unique
 Nomenclature
» Singular matrix: A-1 does not exist, det(A) = 0, rank(A) < n
» Nonsingular matrix: A-1 exists, det(A) non-zero, rank(A) = n
» If rank(A) < n, the matrix is said to rank deficient
 Special Cases
 2x2 matrix
 a11 a12  1 1  a22  a12 
A   A   a 
 21 22 
a a det( A )  21 a11 

 Diagonal matrix
a11 0  0 1 / a11 0  0 
0 a  0  0 1 / a22  0 
A 22   A 1   
         
   
0 0  ann   0 0  1 / ann 

 Product of square matrices

(AB PQ ) 1  Q1P 1 B 1A 1
 Gauss-Jordan Elimination
 Method to compute A-1 using row operations
 Form augmented matrix
  1 1 2   1 1 2 1 0 0
 
A   3  1 1  A  A I    3  1 1 0 1 0
~

 1 3 4  1 3 4 0 0 1

 Eliminate first entry in last two rows

 1 1 2 1 0 0    1 1 2 1 0 0
3  3(1)  1  3(1) 1  3(2) 0  3(1) 1  3(0) 0  3(0)   0 2 7 3 1 0
   
  1  1 3 1 42 0 1 00 1  0   0 2 2  1 0 1
 Gauss-Jordan Elimination
 Eliminate x2 entry from third row

 1 1 2 1 0 0   1 1 2 1 0 0
 0 2 7 3 1 0 0 2 7 3 1 0 
   
0  0 2  2 2  7  1  3 0  1 1  0  0 0  5  4  1 1

 Make the diagonal elements unity

 (1) 1 2 1 0 0  1  1  2  1 0 0 
 0 0.5 ( 2) 0 .5( 7 ) 0.5(3) 0.5 (1) 0   0 1 3.5 1.5 0.5 0 
   
 0 0  0.2(5)  0.2(4)  0.2(1)  0.2(1) 0 0 1 0.8 0.2  0.2
 Gauss-Jordan Elimination cont.
 Eliminate first two entries in third column
1  2(0)  1  2(0)  2  2(1)  1  2(0.8) 0  2(0.2) 0  2(0.2)  1  1 0 0.6 0.4  0.4
 0 1  3.5( 0) 3.5  3.5(1) 1.5  3.5 ( 0.8) 0.5  3.5 ( 0.2) 0  3.5( 0.2)   0 1 0  1.3  0.2 0.7 
   
 0 0 1 0.8 0.2  0.2  
 0 0 1 0.8 0.2  0.2

 Obtain identity matrix

1  1  1 0 0.6  (1.3) 0.4  (0.2)  0.4  0.7 1 0 0  0.7 0.2 0.3 
0
 1 0  1.3  0.2 0.7   0 1 0  1.3  0.2 0.7   I A 1
    
0 0 1 0.8 0.2  0.2  0 0 1 0.8 0.2  0.2

 Matrix inverse
 0.7 0.2 0.3 
A 1    1.3  0.2 0.7 
 0.8 0.2  0.2
 Gauss-Jordan Elimination cont.
 Verify result
 1 1 2  0.7 0.2 0.3 
C  AA 1   3  1 1    1.3  0.2 0.7 
 1 3 4  0.8 0.2  0.2
c11  (1)( 0.7)  (1)( 1.3)  (2)(0.8)  1
c12  (1)(0.2)  (1)( 0.2)  (2)(0.2)  0
c12  (1)(0.3)  (1)(0.7)  (2)( 0.2)  0

1 0 0
C  AA 1  0 1 0  I
0 0 1
 Using the Matrix Inverse
 Linear algebraic equation system: Ax = b
» Assume A is a non-singular matrix
» Solution
Ax  b A 1Ax  A 1b Ix  A 1b x  A 1b
 Example
1 2 1 
3 5 x  2
   
1 1  5  2  1  5  2   5 2 
A        
det( A)  3 1   1  3 1   3  1
1  5 2  1  1
xA b     
 3  1 2  1 
 Ill-Conditioned Matrices
 Matrix inversion: Ax = b  x = A-1b
» Assume A is a perfectly known matrix
» Consider b to be obtained from measurement with some
uncertainty
 Terminology
» Well-conditioned problem: “small” changes in the data b
produce “small” changes in the solution x
» Ill-conditioned problem: “small” changes in the data b
produce “large” changes in the solution x
 Ill-conditioned matrices
» Caused by nearly linearly dependent equations
» Characterized by nearly singular A matrix
» Solution is not reliable
» Common problem for large algebraic systems
 Ill-Conditioned Matrix Example
 Example
0.9999  1.0001  1 
 1  x 
  1  1  e 

» e represents measurement error in b2

» Two rows (columns) are nearly linearly dependent
 Analytical solution
 0.5  5000.5e 
x 
  0.5  4999.5e 

 10% error (b2 = 0.1)

 0.5  (5000.5)(0.1)  500.55
x   
  0.5  ( 4999.5)( 0.1)   499.45