Unit 15 Matrix Operations

Recall that:

is said to be an m n matrix, where m n is called the dimension of A. That is, a matrix whose dimension is m n has m rows and n columns. and also that: the (i, j)-entry of A, denoted by aij , is the entry (i.e., number) in row i and column j of matrix A. That is, for the (i, j)-entry, i is the row number, and j is the column number, where the rows are numbered 1 to m, from the top down, and the columns are numbered 1 to n, from left to right. Denition 15.1. Let A be any m n matrix, and let k and t be integers with 1 k n and 1 t m. We use ak to denote the k th column of matrix A, and at to denote the tth row of matrix A. For instance, for A as a12 a22 2 a = a32 . . . am2 shown above, we have: , a3 = a31 a32 a33 a3n

A = (aij ) =

a11 a21 a31 . . .

a12 a22 a32 . . .

a13 a23 a33 . . .

am1 am2 am3

a1n a2n a3n . . . . . . amn

Notice: In ak , all entries have the same second subscript k (because all are in the same column), while in at , all entries have the same rst subscript t (because all are in the same row).

Denition 15.2. Two matrices, A and B, are equal, written A = B, if they have the same dimension and their corresponding entries are equal. That is, A and B are equal if both are m n matrices (for the same values of m and n) and if aij = bij for each i and j. Example 1. State whether matrices A and B are equal. (a) A = (b) A = 1 2 3 4 0 6 1 0 3 5 1 2 1 2 3 4 5 6 1 0 0 1 B= B= 1 2 3 4 0 6 1 0 3 5 1 2 1 4 2 5 3 6 0 1 0

(c) A =

B= 1 0 B= 0

(d) A = Solution: (a) A =

1 2 3 4 0 6

=B=

1 2 3 4 0 6

Since A and B both have dimension 2 3 and aij = bij for each pair (i, j), they are equal matrices. (b) A = 1 0 3 5 1 2 =B= 1 0 3 5 1 2

Although A and B are both 2 3 matrices, with many of their entries identical, there is a combination ij for which aij = bij (i.e. a23 = 2 whereas b23 = 2). Therefore, A and B are not equal matrices. 1 2 3 4 5 6 1 4 =B= 2 5 3 6

(c) A =

Here, A has dimension 2 3, whereas B has dimension 3 2, so they cannot be equal matrices, no matter how similar their entries may be. 2

(d) A =

1 0 0 1

Again, A and B do not have the same dimension (A is 2 2 while B is 3 2), so they are not equal.

1 0 =B= 0 1 0 0

Addition, Subtraction and Scalar Multiplication of Matrices Denition 15.3. If A and B are matrices with the same dimension then the sum of A and B, written A + B, is obtained by adding corresponding entries. i.e. If C = A + B then cij = aij + bij Notice: Addition of two matrices, A and B, is only dened when A and B have the same dimension. Example 2. Find the sum of matrices A and B if possible. (a) A = (b) A = Solution: (a) Since A is a 2 3 matrix and B is also a 2 3 matrix, the sum A + B is dened. We add corresponding entries to get A+B = = = 2 1 3 0 2 5 + 1 5 0 2 4 6 2 1 3 0 2 5 2 3 1 4 B= B= 1 5 0 2 4 6

1 3 2 2 1 3

(2 + 1) (1 + 5) (3 + 0) (0 2) (2 + 4) (5 6) 3 4 3 2 6 1

(b) This time, A is a 2 2 matrix while B is a 2 3 matrix. Matrix addition can only be performed when the matrices to be added have the same dimension, so in this case A + B is not dened.

Denition 15.4. Let c be any scalar and A be any matrix. Then the scalar multiple cA is obtained by multiplying each entry in A by the scalar c. i.e. if B = cA then bij = c aij for every i and j 1 2 3 5 =

Example 3. For A = Solution: We have 3A = 3 Similarly, (2)A = (2) 1 2 3 5

nd 3A and 2A. 31 32 3 3 3 (5) 3 6 . 9 15

1 2 3 5

(2) 1 (2) 2 (2) 3 (2) (5)

2 4 . 6 10

Denition 15.5. The negative of any matrix A, denoted A, is dened to be (1)A. That is, the negative of a matrix is obtained by multiplying the matrix by the scalar 1, which has the eect of changing the sign of each entry in the matrix. For instance, for A = we have A = (1) 1 0 2 3 1 0 2 3 = 1 0 . 2 3

Denition 15.6. Subtraction of matrices is dened as addition of the negative of the matrix. That is, the matrix dierence A B is dened as A B = A + (B). Notice: Of course, A and B must have the same dimension in order for the subtraction A B to be dened, since this is necessary in order for the addition A + (B) to be dened. That is, A and B must have the same dimension, and the dimension of B is, by denition, the same as the dimension of B. Notice also: If C = A B for some matrices A and B both having dimension m n, then C = A + (B), so we have cij = aij + (bij ) = aij bij . That is, 4

the dierence matrix A B is obtained by subtracting each entry bij from the corresponding entry aij . Example 4. Given A and B as follows, nd 3A B. A= Solution: 3A B = 3 1 2 3 4 1 2 2 0 = 3(1) 3(2) 3(3) 3(4) 1 2 2 0 1 2 3 4 B= 1 2 2 0

31 62 9 (2) 12 0 2 4 11 12

Denition 15.7. A matrix in which all entries are 0 is called a zero matrix, and is usually denoted by 0. There are 0 matrices of every possible dimension. 0 0 , 0 0 0 0 0 , and 0 0 0

For instance,

0 0 0

are all zero matrices.

They are, respectively, the 2 2 zero matrix, the 2 3 zero matrix and the 1 3 zero matrix.

Matrix Multiplication Recall the dot product of two vectors in For instance, (1, 2, 1) (3, 4, 5) = (1)(3) + (2)(4) + (1)(5) = 6 and (2, 0, 1) (1, 5, 3) = (2)(1) + (0)(5) + (1)(3) = 5 5
n

Denition 15.8. Let A be any m n matrix. Then the matrix product AB is only dened where B is an n k matrix (for any integer k). That is, in order to multiply A B, the number of rows in matrix B must be equal to the number of columns in matrix A. For any m n matrix A and any n k matrix B, the product matrix AB is the m k matrix whose (i, j)-entry is obtained by taking the dot product of the vectors corresponding to the row vector ai and the column vector bj . That is, we take the n-vector whose components are the entries of row i of matrix A, i.e.,ai , and the n-vector whose components are the entries of column j of matrix B, i.e., bj , and nd the dot product of these 2 n-vectors. The resulting number is the (i, j)-entry of the product matrix AB. We (somewhat sloppily) use ai and bj to denote these n-vectors, as well as the corresponding row vector and column vector, which allows us to write that: For C = AB, cij = ai bj Thus, we have cij = ai1 b1j + ai2 b2j + . . . + ain bnj .

cij

mk

ai

mn

j b

nk

Notice that: 1. The reason that we must have the number of columns of A equal to the number of rows of B in order to form the matrix product AB is so that these dot products will be dened, because dot product is only dened between vectors of the same space, i.e., between vectors with the same number of components. So the ith row of A must contain the same number of entries as the j th column of B. 2. We have m rows in matrix A and k columns in matrix B, and we do one of these dot products for each combination of [a row from A] and [a column from B]. Thus, the product matrix has the same number of rows as matrix A, and the same number of columns as matrix B. That is, if A is m n and B is nk, then we get AB having dimension mk, as stated in the denition.

Example 5. If the dimension of matrix A is 2 3 and the dimension of matrix B is 3 6 then what is the dimension of matrix C = AB? Solution: Notice that the matrix product AB is dened, since the number of rows of B is the same as the number of columns of A. Thus, C is dened and its dimension is given by C = AB (2 3 )( 3 6) (2 6)

That is, C has the same number of rows as A, and the same number of columns as B, so C has dimension 2 6. Example 6. Perform the following matrix multiplications, if possible: 1 2 1 2 1 (a) 3 1 1 3 2 2 4 1 2 3 3 (b) 1 2 1 1 3 1 2 2 (c) 1 2 1 1 3 2 1 2 2 1 3 1

Solution: 1 2 1 1 3 2 1 2 3 1 = 2 4 = (1)(1) + (2)(3) + (1)(2) (1)(2) + (2)(1) + (1)(4) (1)(1) + (3)(3) + (2)(2) (1)(2) + (3)(1) + (2)(4) 9 4 14 7

(a)

Notice: We are multiplying a 23 matrix times a 32 matrix, so the product matrix has dimension 2 2. 1 2 3 3 (1)(3) + (2)(1) + (3)(2) 1 1 2 1 1 = (1)(3) + (2)(1) + (1)(2) = 1 3 1 2 2 (3)(3) + (1)(1) + (2)(2) 4

(b)

Notice: The result of multiplying a 3 3 matrix times a 3 1 matrix is a 3 1 product matrix.

1 2 1 1 3 2 (c) (2 3 )

1 2 2 1 3 1 ( 2 3)

is not dened.

Here, we are asked to multiply a 2 3 matrix times a 2 3 matrix. This matrix product is not dened, since the number of columns in the rst marix (3) is not equal to the number of rows in the second matrix (2). Denition 15.9. Any matrix which has the same number of rows as columns is called a square matrix. That is, if A is a square matrix, then A has dimension n n for some value of n. This number, n, is called the order of the square matrix. For instance, a 2 2 matrix is a square matrix of order 2. Denition 15.10. Powers of a Matrix: Let A be any square matrix. Then we dene: A1 A2 A3 . . . An = A = (A)(A) = (A)(A)(A) = (A)(A)(A) (A), n times

That is, we get powers of a matrix by multiplying the matrix by itself repeatedly. We use an exponent on a matrix to denote the number of copies of the matrix which are being multiplied together, just as exponents are used in other contexts. That is, just as 23 = (2)(2)(2) = 8 and x3 = (x)(x)(x), we have A3 = (A)(A)(A). Notice that in order to multiply A times A, we must have the number of rows of A (the second matrix in the product) equal to the number of columns of A (the rst matrix in the product). This is why An is only dened when A is a square matrix. Notice also: In order to see that a matrix power is well dened (i.e., has one and only one meaning) we use the fact that matrix multiplications is associative, so that [(A)(A)](A) = (A)[(A)(A)]. That is, whether we calculate A3 as (A2 )(A) or as (A)(A2 ), we get the same result.

Example 7. Given A = Solution: A2 = A A =

1 2 , calculate A2 and A3 . 3 1 1 2 3 1 = 5 0 0 5 1 2 3 1 = 5 10 15 5

1 2 3 1

A3 = A A A = A2 A =

5 0 0 5

Exercise: For the matrix given here, show that A3 = A A2 gives the same result. Transpose of a Matrix Denition 15.11. The transpose of an m n matrix A, written AT , is the n m matrix obtained by interchanging the rows and the columns of A. That is, the ith row of A is the ith column of AT (so that also the j th column of A becomes the j th row of AT ). Thus, if B = AT , then bi = ai so that also bj = aj . T a11 a21 am1 a11 a12 a13 a1n a12 a22 am2 a21 a22 a23 a2n a23 am3 = a13 . . . . . . . . . . . . am1 am2 am3 amn a1n a2n amn We see that the (i, j)-entry of A becomes the (j, i)-entry of AT , so that if B = AT , then bij = aji . Example 8. Given A, nd AT for each of the following. 2 1 3 2 (a) A = (b) A = 4 7 5 7 9 3 Solution: (a) We interchange the rows and columns of A. That is, the rst row of A becomes the rst column of AT and the second row of A becomes the second column of AT . We get: A =
T

3 2 5 7 9

3 5 2 7

(b) Again, the roles of row and column are switched, so that the 3 rows of 2 entries each in the 3 2 matrix A become 3 columns of 2 entries each in the 2 3 matrix AT . T 2 1 We get:AT = 4 7 = 9 3 2 4 9 1 7 3

Denition 15.12. If A is a square matrix of order n (with entries aij ), then the entries aij for which i = j form the main diagonal of A. a11 a21 a31 . . . an1 2 5 8 the a12 a22 a32 . . . an2 a13 a23 a33 . . . an3 . . . a1n a2n a3n . . . ann

A=

1 4 For instance, for B = 7 we see that 1, 5 and 9 are

3 6 9 entries on the main diagonal.

Denition 15.13. A square matrix of order n whose only non-zero entries are on the main diagonal is called a diagonal matrix. 3 0 0 1 0 For instance, and 0 2 0 are both diagonal matrices, be0 2 0 0 5 cause in each case, aij = 0 whenever i = j.

Denition 15.14. A diagonal matrix of order n which has all of the entries on the main diagonal equal to 1 (and all other entries equal to 0, since it is diagonal) is called the Identity Matrix of order n and is denoted by I, or In when dimension is not otherwise apparent. The n n identity matrix has the important property that if A is n n then IA = AI = A 10

For instance,

1 0 0 1

= I2

1 0 0 and 0 1 0 = I3 are identity matrices. 0 0 1 a b c d

Example 9. Show that IA = AI = A, for A =

Solution: Note: Here, it is clear from the context that I refers to I2 , since A is a 2 2 matrix and we need both IA and AI to be dened operations. IA = 1 0 0 1 a b c d AI = a b c d 1 0 0 1

1(a) + 0(c) 1(b) + 0(d) 0(a) + 1(c) 0(b) + 1(d) a b c d

a(1) + b(0) a(0) + b(1) c(1) + d(0) c(0) + d(1) a b c d

The following properties of operations on matrices are extremely important and should be studied thoroughly. Many exercises should be completed so that they become second nature. Properties (and Rules) of Operations for Matrices 1. A + B = B + A for any matrices A and B of the same dimension. (i.e., matrix addition is commutative) 2. A + (B + C) = (A + B) + C for any matrices A, B and C, all with the same dimension. (i.e., matrix addition is also associative) 3. A + 0 = A for any matrix A, where of course 0 denotes the 0-matrix whose dimension is the same as the dimension of A. 4. A + (A) = 0 for any matrix A (again, 0 is the zero matrix with the same dimension as A here). 5. A(BC) = (AB)C for any matrices A, B and C of appropriate dimensions to perform this multiplication. i.e., if A is m n and C is p q, then B must be n p. (i.e., as previously stated, matrix multiplication is associative) 11

6. AI = A for any m n matrix A, where I has order n and also IA = A for any m n matrix A, where I has order m. 7. A(B + C) = AB + AC for any matrices A, B and C, where A is m n and B and C are both n p. and also (B + C)A = BA + CA for any matrices A, B and C, where B and C are both m n and A is n p. (i.e., matrix multiplication is distributive over matrix addition) 8. a(B + C) = aB + aC for any scalar a and any matrices B and C of equal dimension. (i.e., scalar multiplication of matrices is distributive over matrix addition) 9. (a + b)C = aC + bC for any scalars a and b and any matrix C. (i.e., scalar multiplication of matrices is also distributive over scalar addition) 10. (ab)C = a(bC) for any scalars a and b and any matrix C. 11. 1A = A for any matrix A. Note: 1 is the scalar 1, of course.

12. A0 = 0 for any matrix A, where the two zero matrices are of appropriate dimensions. and also 0A = 0 for any matrix A, where again the two zero matrices are of appropriate dimensions. 13. a0 = 0 for any scalar a and any zero matrix. 14. a(AB) = (aA)B = A(aB) for any scalar a and any matrices A and B of appropriate dimensions to form the matrix product AB. (i.e., scalar multiplication of a matrix product can either be performed after the matrices are multiplied or else be performed before the matrix multiplication, by multiplying either of the matrices in the product by the scalar.) 15. (A + B)T = AT + B T for any matrices A and B of the same dimension. (i.e., matrix transposition is distributive over matrix addition) 16. (AB)T = B T AT for any matrices A and B of appropriate dimensions to be multiplied. (i.e., matrix transposition is distributive over matrix multiplication, but the order of multiplication is reversed)

12

It is worth looking at an example, to demonstrate this last property as well as the reason for this reversal in the order of multiplication. 1 1 Example 10. For A = 1 2 and B = 1 3 1 0 , nd (AB)T . 2 1

Solution: According to property 16, we can either nd the matrix product AB and then take its transpose, or else take the transposes of A and B and then nd the product (B T )(AT ). By the rst approach, we have: 1 1 1 0 AB = 1 2 2 1 1 3

3 1 1+2 01 = 1 + 4 0 2 = 5 2 7 3 1+6 03 3 5 7 1 2 3

So (AB)T =

Notice: A is 3 2 and B is 2 2, so AB is 3 2 and (AB)T is 2 3. By the second approach, we have AT = 1 1 1 1 2 and B T = . 1 2 3 0 1 Notice that AT is 2 3 and B T is 2 2. The matrix product (AT )(B T ) is not dened, because the dimensions are not appropriate for multiplying these matrices. However, the matrix product (B T )(AT ) is dened (this is true whenever the matrix product AB is dened). In this case, we can multiply a 2 2 times a 2 3, and the result will be a 2 3 product matrix. Even if the dimensions were not wrong (without the reversal), because we have interchanged the roles of rows and columns in taking the transposes of the matrices, we must reverse the order of multiplication, in order to be doing the same dot product calculations as we did using the rst approach.

We get (B T )(AT ) = =

1 2 0 1

1 1 1 1 2 3 = 3 5 7 1 2 3

1+2 1+4 1+6 01 02 03

We see that this matrix is the same as the one found by the rst approach. That is, we see that (B T )(AT ) = (AB)T .

13

It is equally important to recognize the properties that do not hold for matrix operations. Most notably, matrix multiplication is not commutative. That is, in general, AB = BA, even when both of these matrix products are dened and they have the same dimension. First of all, notice that neither AB nor BA might be possible. (i.e., A and B might not have appropriate dimensions for either product to exist.) And even if one of these matrix products exists, the other may not. As well, if both AB and BA exist, they may have dierent dimensions, and therefore cannot be equal. For instance, let A be a 4 2 matrix, B be a 3 2 matrix and C be a 2 3 matrix. Then neither AB nor BA exists (i.e., these matrix products are not dened, because the number of columns of A is not equal to the number of rows of B, and also the number of columns of B does not equal the number of rows of A). The matrix product AC exists, but the matrix product CA does not exist (since the number of columns of A is equal to the number of rows of C, but the number of columns of C is not equal to the nubmer of rows of A). And although both BC and CB exist, BC is a 3 3 matrix, while CB is a 2 2 matrix. However, even if A and B are square matrices of the same order, so that AB exists and also BA exists, and, in addition, AB and BA have the same dimension, it still will not usually be true that these are equal matrices. Example 11. For A = Solution: AB = 1 1 2 1 1 2 1 2 1 2 1 2 1 1 2 1 = 11 2+2 21 4+2 1+4 1+2 1 + 4 1 + 2 = 0 4 1 6 5 3 3 1 = AB 1 1 2 1 and B = 1 2 , show that AB = BA. 1 2

BA =

14