MA 35100 LECTURE NOTES: MONDAY, JANUARY 25

Matrix Algebra You are probably familiar with adding vectors, and multiplying them by scalars: (x1 , y1 ) + (x2 , y2 ) = (x1 + x2 , y1 + y2 ) , (Recall that a scalar k matrices) in Rn : a1 a2 . . . an k (x, y ) = (k x, k y ) . is simply a real number.) The same is true for arbitrary vectors (column b1 b2 . . . bn a1 + b1 a2 + b2 . . . an + bn , k a1 a1 ka a2 .2 k = . . an k an .

We make the same denition for arbitrary n m matrices. Denition. Consider two n m matrices A and B : a11 a12 a1m b11 a21 a22 a2m b21 A= . , B = . . . . . . . . . . . . . . an1 an2 anm

b12 b22 . . .

.. .

b1m b2m . . . bnm

bn1 bn2

The sum of A and B is dened as that n m matrix with entries aij + bij : a11 + b11 a12 + b12 a1m + b1m a21 + b21 a22 + b22 a2m + b2m A+B = . . . . .. . . . . . . . an1 + bn1 an2 + bn2 anm + bnm The scalar multiplication of A with a scalar k is dened as that n m matrix with entries k aij : k a11 k a12 k a1m k a21 k a22 k a2m kA = . . . . .. . . . . . . . k an1 k an2 k anm Matrix Multiplication: Vectors and Vectors We generalize the concept of scalar multiplication to larger matrices. We do so by beginning with a familiar denition. Denition. Let w be a row vector and x be a column vector, both with m components. Explicitly, write x1 x2 w = w1 w2 wm and x= . . . . xm
1

The dot product of w with x is the scalar w x = w1 x 1 + w2 x 2 + + wm x m . We extend the denition of dot product to row or column vectors, as long as they have the same number of components. We list a few facts about the dot product which will be helpful to remember: Theorem. Let u, v , and w be either row or column vectors with m components, and let k be a scalar. The following equations hold: (1) Commutativity : v w = w v . (2) Distributivity : (u + v ) w = u w + v w. (3) Associativity : (k v ) w = k (v w). (4) Positivity : v v 0, with equality if and only if v = 0 is the zero vector. (This information can be found in the appendix of the text.) Matrix Multiplication: Matrices and Vectors We continue to extend the denition of matrix multiplication. Denition. Consider an n m matrix A, and let vj denote the j th column of A as an n-dimensional vector. Explicltly, write a11 a12 a1m a1j a21 a22 a2m a2j A= . so that vj = . . . . .. . . . . . . . . . an1 an2 anm anj The matrix multiplication of A with an m-dimensional vector x Rm is dened as the following n-dimensional vector: x1 x2 A x = v1 v2 vm . = x1 v1 + x2 v2 + + xm vm . . . xm Youll note the similarity with the denition of the dot product. Note also that A x is dened only when the number of columns of A matches the dimension of the vector x. As an example, consider the following: 3 1 0 1 A= and x = 1 . 1 2 3 2 The product is dened as Ax = 3 1 1 +1 0 2 +2 1 3 = 3 3 + 0 2 + 2 6 = 1 11 .

Linear Combinations The previous denition motivates a new denition. Denition. A vector b Rn is called a linear combination of the vectors v1 , v2 , . . . , vm Rn if there exist scalars x1 , x2 , . . . , xm R such that b = x1 v1 + x2 v2 + + xm vm .
2

In particular, the product A x is the linear combination of the columns of A with the components of x as coecients. Conversely, given a linear combination b = x1 v1 + x2 v2 + + xm vm , we may write b = A x in terms of the matrices x1 x2 and x= . A = v1 v2 vm . . . xm Note that the product A x is dened in terms of the columns of A. The next statement explains how to express the product in terms of the rows of A. Theorem. Consider an n m matrix A, and let wi denote the ith row of A as an m-dimensional vector. Explicitly, write a11 a12 a1m a21 a22 a2m A= . so that wi = ai1 ai2 aim . . . . . . . . . . . . an1 an2 anm The matrix multiplication of A with an m-dimensional vector x Rm can be expressed as the following n-dimensional vector: w1 x1 w1 x w 2 x2 w 2 x Ax = . . = . . . . . . . . wn xm wn x To see why this is true, we write everything out in detail: A x = x1 v1 + x2 v2 + + xm vm a11 a12 a1m a21 a22 a2m = x1 . + x + + x . . 2 m . . . . . . an1 an2 anm a11 x1 + a12 x2 + + a1m xm w1 x a21 x1 + a22 x2 + + a2m xm w2 x = = . . . . . . an1 x1 + an2 x2 + + anm xm wn x

We use this fact to prove a corollary which we will use repeatedly throughout the course: Theorem. Matrix multiplication preserves linear combinations. Explicitly, let A be an n m matrix, and v = k1 x1 + k2 x2 + + kr xr be a linear combination of vectors in Rm . Then the vector A v is a linear combination of vectors in Rn : A v = k1 A x1 + k2 A x2 + + kr A xr . We explain why this is true. It suces to show the following: Given two vectors x, y Rm and a scalar k R, we have a. A (x + y ) = A x + A y b. A (k x) = k (A x).
3

The reason why this suces is we can express v = x + y in terms of x = kr xr and y = k1 x1 + k2 x2 + + kr1 xr1 , then work inductively. Recall that we may write w1 v w2 v Av = . . . wn v where wj is the j th row of A. To show statement (a), we invoke the distributivity property of the dot product: w1 (x + y ) w1 x + w1 y w1 x w1 y w2 (x + y ) w2 x + w2 y w2 x w2 y A (x + y ) = = = + . . . . . . . . . . . . wn (x + y ) = A x + A y. The proof of statement (b) is similar; we invoke the associativity property: w 1 (k x ) k (w1 x) w1 x w2 (k x) k (w2 x) w2 x A (k x ) = = = k = k (A x) . . . . . . . . . . w n (k x ) k (wn x) wn x Matrix of a Linear System We showed above that a11 a12 a21 a22 . . . . . . we have the matrix product: a1m x1 a11 x1 + a12 x2 + + a1m xm x2 a21 x1 + a22 x2 + + a2m xm a2m . = . . .. . . . . . . . anm xm an1 x1 + an2 x2 + + anm xm . wn x + wn y wn x wn y

an1 an2

As you might guess, we can use this to write systems of linear equations in a compact form: Theorem. Consider the following linear system of n equations in m unknowns: a11 x1 a21 x1 . . . + a12 x2 + a22 x2 . . . + + + a1m xm + a2m xm . . . = b1 = b2 . = . .

an1 x1 + an2 x2 +

+ anm xm = bn

Denote A as the coecient matrix, x as that vector in Rm with the variables as its components, and b as that vector in Rn with the constants as its components. The system can be expressed as the matrix product A x = b i.e. a11 a12 a1m x1 b1 a21 a22 a2m x2 b2 = . . . . . . . . . . . . . . . . . . . an1 an2 anm xm bn A b . Moreover, the system has the augmented matrix