Md1

Lectures 1-6

Matrix Algebra

6.1 Basic Concepts: Matrix Addition, Scalar Multiplication 6.2 Matrix Multiplication 6.3 Linear Systems of Equations: Gauss Elimination 6.4 Rank of a Matrix. Linear Independence. Vector Space 6.5 Solutions to Linear Systems: Existence, Uniqueness 6.6 Determinants. Cramers Rule 6.7 Inverse of a Matrix: Gauss-Jordan Elimination 6.8 Vector Spaces, Inner Product Spaces, Linear Transformations

Md2
G Motivation:

Linear Algebra
Theory and application of linear systems of equations (Linear Systems) Linear Transformations Eigenvalue problems

G Applications:
Electrical networks Frameworks in mechanics Curve fitting and other optimisation problems Systems of differential equations Processes in statistics

G Utility:
Matrix: rectangular array of numbers, considered as a single object Vectors & Matrices: very compact, elegant and powerful shorthand Enabling: simplified analysis and effective modelling of systems Highly suitable for computer implementation

Md3

Matrix Notation

G Notation: rectangular array of numbers (or functions) enclosed in brackets

elements row

2 5

0.4 32

8 , 0

6 1,

[a1

a2

a3 ],

a c

b , d

ex 2x e

3x x2

column

column vector

row vector

square matrices

G General Matrix form:

A = a jk

[ ]

a11 a 21 = . a m1

a12 a22 . am 2

... a1n ... a2 n ... . ... amn

m n matrix A = a jk , m rows and n columns double subscript notation, a jk , row j and column k square matrix when m = n, main diagonal elements aii

[ ]

Md4

Matrix Transposition & Equality

5 2 a = [5 3 1], b = , a T = 3, 4 1 b T = [2 4 ]

G Row, column vectors - transposition:

m n matrix A has n m transpose AT rows of matrix A ( AT ) columns of AT ( A)

G Matrix transposition:

5 4 5 8 1 T A = [a jk ], AT = [akj ], e.g. A = , A = 8 0 4 0 0 1 0

G Symmetric and Skew-symmetric matrices:

Symmetric AT = A, Skew symmetric AT = A

G Equality of matrices:
Two matrices are equal iff (if and only if) they have the same size and corresponding elements are equal.

Md5

Matrix Addition
A+B only defined for matrices A, B of the same size A+B obtained by adding corresponding elements.

G Matrix Addition:

5 1 0 1 5 3 4 6 3 , , A= B = A B + = 3 1 0 3 2 2 0 1 2
G Addition Properties:

For matrices of the same size ( m n) : A + B = B + A (commutativity) (U + V ) + W = U + (V + W ) = U + V + W A+0 = A A + ( A) = 0 (identity for addition) (additive inverse) (associativity)

Md6
G Definition:

Scalar Multiplication
Product of m n matrix A = [a jk ] and scalar c is m n matrix : cA = [ca jk ] obtained by multiplying each element in A by c.

G Properties:

c( A + B) = cA + cB (c + k ) A = cA + kA

(distributivity -1) (distributivity - 2)

c( kA) = (ck ) A = ckA (associativity) 1A = A (identity for scalar multiplication)

G Properties of Transposition:

( A + B)T = AT + BT (cA)T = cAT

(term by term addition)

(scalar multiplication)

Md7
G Definition:

Matrix Multiplication
Product C = AB of m n matrix A = [a jl ] and n p matrix B = [ b lk ] is : m p matrix C = [c jk ] with elements : c jk = a jl b lk = a j1b1k + a j2 b 2 k + ... + a jn b nk ; j = 1,..., m; k = 1,..., p
l =1 n

i.e. jk

th

element of C is :

the dot product of jth row vector of A and k th column vector of B

G Differences from multiplication of numbers:

Matrix multiplication is not commutative, AB BA in general AB = 0 does not necessarily imply A = 0 or B = 0 or BA = 0 AC = AD does not necessarily imply C = D (even when A 0)

Md8

Matrix Multiplication AB=C

A B C k

a11 a12 a a22 21 . . a j1 a j 2 . . . . . . am1 am 2

... ... ... ... ... ... ... ...

mn

a1n a2 n b11 b12 . b21 b22 . . a jn . . . . . . . bn1 bn 2 . amn

... b1k ... b2 k ... . ... . ... . ... bnk

... ... ... ... ... ...

n p

c11 c12 b1 p c21 c22 . b2 p . . c j1 c j 2 = . . . . . . bnp . . cm1 cm 2

... ... ... ... ... ... ... ...

c1k c2 k . c jk . . . cmk

... ... ... ... ... ... ... ...

c1 p c2 p . c jp . . . cmp

m p

Md9

Properties of Matrix Multiplication

Importance of ordering of factors premultiplication (multiply from left) postmultiplication (multiply from right)

G Properties:

( kA) B = k ( AB) = A( kB) = kAB

(scalar k )

A( BC ) = ( AB)C = ABC (associativity) ( A + B)C = AC + BC (distributivity) C( A + B) = CA + CB

G Transpose of a Product:

(distributivity)

( AB)T = BT AT note products of transposed factors in reverse order

Md10
G Triangular Matrices:

Special Matrices

Upper triangular matrices (zero entries below diagonal) Lower triangular matrices (zero entries above diagonal)

G Diagonal Matrices:

Non - zero entries only on the main diagonal A Scalar matrix is a diagonal matrix with all diagonal elements equal : For any square matrix A and Scalar matrix S : AS = SA = cA Identity matrix I is a Scalar matrix with diagonal elements = 1.

Md11

Scalar (Dot) Product of Vectors

b1 . n ... a n ] . = a k b k = a1b1 + a 2 b 2 + ... + a n b n . k =1 b n

G Definition of Dot Product:

a b = [a 1

G Product in terms of Row and Column vectors:

C = AB; c jk = a j b k = ( jth row of A ) ( k th column of B) a1 b1 a1 b 2 a b a b 2 1 2 2 = . . a b a b 2 m m 1 ... a1 b p ... a 2 b p ... . ... a m b p

C m p = A m n Bn p

Md12

Motivation of Matrix Multiplication

y1 = a11x1 + a12 x 2 y 2 = a 21x1 + a 22 x 2

G Linear Transformations:

x1 = b11w1 + b12 w 2 x 2 = b 21w1 + b 22 w 2

b12 c11 ; C= b 22 c 21 c12 . c 22

G Composite Transformation:

b11 y1 x1 w1 a11 a12 y = ; x = ; w = ; A = ; B= y 2 x 2 w 2 a 21 a 22 b 21

y = Ax; x = Bw; y = Cw = ABw, where C = AB
G Higher dimensions:

y = Ax; x = Bw; y = Cw = ABw, where C = AB

Md13

Applications of Matrix Multiplication

w M W F S b j

G Weight watching. Matrix times vector:

0 0.5 750 1.0 1.0 0.75 0 300 600 400 = 750 0 0.5 1.0 900 1.5 1.0 0.5 1300

G Computer production. Matrix times matrix:

Cost per PC, A; Production in year 2000, B PC1 PC2 Quarters (1 4) Quarters (1 4) Raw 0.3 0.5 2.2 1.8 1.6 1.9 PC1 4 1 2 3 C = AB = Lab 0.8 1.3 = 5.8 4.7 4.2 5.0 PC2 2 3 2 2 Misc 0.7 0.9 4.6 3.4 3.2 3.9

Md14

Applications (continued)
% To

G Stochastic Matrix. Markov process. Powers of a matrix:

Res 30 0.8 0.1 0.1 transition probs x = Com 20 , Transition matrix, A = From 0.1 0.7 0.2 for 5 - year 50 Ind intervals 0 0.1 0.9 x is 1998 % land use in city of 50 sq miles. Find states in 2003, 2008 & 2013

0.8 0.1 0.1 y = x A = [30 20 50] 0.1 0.7 0.2 = [26 22 52] 0 0.1 0.9
T T

z T = y T A = ( x T A) A = x T A2 = [23.0 23.2 53.8]

u T = z T A = ( x T A2 ) A = x T A3 = [20.72 23.92 55.36]

Md15

Rotation Matrices
x1 y1 cos sin Let A = , x = , y = sin cos x 2 y 2 Then, y = Ax represents a counterclockwise rotation of the cartesian x1x 2 coordinate system in the plane about the origin. is the angle of rotation. y1 = x1 cos x 2 sin
x2 y2 x2cos y2 x2 x2 x1 y1 x1 x1cos P: (y1, y2) P: (x1, x 2) y1 x1sin x1 x2sin

G Rotation in the x-y plane:

y 2 = x1 sin + x 2 cos whence, y = Ax cos 2 sin 2 A2 = sin 2 cos 2 cos n sin n An = sin n cos n

Md16
G

Linear Systems of Equations

A linear system of m equations in n unknowns x1 , ... , xn : a11 x1 + ... + a1n xn = b1 a21 x1 + ... + a2 n xn = b2 ... am1 x1 + ... + amn xn = bm

Linear System:

e.g. 2 equations in 3 unknowns : 5x1 + 2 x2 x3 = 4 x1 4 x2 + 3 x3 = 6

Augmented matrix

Matrix Form:

Ax = b
... a1n ... a 2 n , ... . ... a mn

a11 a12 a a 22 21 A= . . a m1 a m 2

x1 a11 b1 x 2 a b 2 = 21 x = . , b = ; A . . . a bm m1 x n

... a1n ... a 2 n ... . ... a mn

b1 b2 . bm

Md17
G

Geometric Interpretation
( x1 , x2 ) a11 x1 + a12 x2 = b1 a21 x1 + a22 x2 = b2
Each equation represents a straight line in the plane

Coordinates in the plane for m = n = 2:

Three possible cases:

(a) No solution if the lines are parallel (b) Precisely one solution if the lines intersect (c) Infinitely many solutions if the lines coincide
y (a) x+y=1 x+y=0 y (b) x+y=1 x-y=0 y (c) x+y=1 2x+2y=2

Md18
G

Gauss Elimination
Equations

Solve the linear system:

Augmented Matrix A
x3 = 0 = 80

Pivot 1

x1 x2 + x3 = 0 x1 + x2 10 x2 + 25 x3 = 90 20 x1 + 10 x2

Eliminate

1 1 1 1 1 1 0 10 25 20 10 0

0 0 90 80

Elimination of: x1

x1 x2 +

x3 = 0 0=0

10 x2 + 25 x3 = 90 30 x2 20 x3 = 80

1 1 1 0 0 0 0 10 25 0 30 20

0 0 90 80

Row 2 + Row 1

Row 4 - 20 Row 1

Md19
G

Gauss Elimination - 2
x1 x2 + x3 = 0

Partial pivoting:

Pivot 10 Eliminate

10 x2 + 25 x3 = 90 30 x2 20 x3 = 80 0=0

1 1 1 0 10 25 0 30 20 0 0 0

0 90 80 0

Elimination of: x2

x1 x2 +

x3 = 0

10 x2 + 25 x3 = 90 95 x3 = 190 0=0

1 1 1 0 10 25 0 0 95 0 0 0

0 90 190 0

Row 3 - 3 Row 2

Md20
G

Gauss Elimination - 3

Back substitution:

95 x3 = 190 10 x2 + 25 x3 = 90 x1 x2 +
G

x3 = ( 190) /( 95) = 2 x2 = (90 25 x3 ) / 10 = 4 x1 = x2 x3 = 4 2 = 2

x3 = 0

Unique solution in this case:

x1 2 x1 = 2, x2 = 4, x3 = 2; x = x2 = 4 . 2 x3

Md21
G

Elementary Row Operations

Interchange of two equations Addition of a constant multiple of one equation to another equation Multiplication of an equation by a nonzero constant c

Elementary Operations for Equations:

Elementary Row Operations for Matrices:

Interchange of two rows Addition of a constant multiple of one row to another row Multiplication of a row by a nonzero constant c

Row-equivalent Systems:
A linear system S1 is row-equivalent to linear system S2 if S1 can be obtained from S2 by finitely many elementary row operations. Row-equivalent linear systems have the same sets of solutions. Whence justification for Gauss elimination procedure.

Md22
G

Three possible cases of systems

3.0 2.0 2.0 5.0 8.0 0.6 1.5 1.5 5.4 2.7 1.2 0.3 0.3 2.4 2.1
3.0 2.0 2.0 5.0 8.0 0 1.1 1.1 4.4 1.1 0 0 0 0 0
3.0 2.0 2.0 5.0 8.0 0 1.1 1.1 4.4 1.1 0 1.1 1.1 4.4 1.1
Back Substitution:

(a) Infinitely many solutions exist:

r2 - 0.2*r 1 r3 - 0.4*r 1

x2 = 1 x3 + 4 x4 , where x3 , x4 arbitrary
r3 + r 1

x1 = (8 2 x2 2 x3 + 5 x4 ) / 3 = 2 x4

(b) Unique solution exists:

1 1 2 2 3 1 1 6 1 3 4 4

1 1 2 2 2 7 12 r2 + 3*r1 0 r3 - r1 0 2 2 2

2 1 1 2 0 2 7 12 r3 - r2 0 0 5 10

Back Subst.

x3 = 2, x2 = 1, x1 = 1

Md23
G

No solution exists
3 2 1 3 2 1 1 0 r2 2 3 r1 6 2 4 6 r3 2r1 3 2 1 3 0 1 1 2 3 3 0 2 2 0 r3 6r2 3 2 0 1 3 0 0 1
1 3

(c) No solution exists:

3 2 0 12

Inconsistency
The last row effectively asserts that 0 = 12

Md24
G

Echelon Form
a11 x1 + a12 x2 + .......... + a1n xn = b1 c22 x2 + .......... + c2 n xn = b 2 ... krr xr + ... + krn xn = b r 0=b r +1 ... 0=b m where r m ( and a11 0, c22 0, ..., krr 0).

Reduced system has form:

, ... , b 0 (a) No solution if r < m and one of b r +1 m , ... , b = 0 (if present) (b) Precisely one solution if r = n and b r +1 m , ... , b = 0 (if present) (c) Infinitely many solutions if r < n and b r +1 m