Matrix Theory

Matrix Operations
Just like vectors of the same size, matrices can be added together and multiplied
by scalars. Matrix-Matrix multiplication is more interesting. Recall that we
have defined matrix-vector multiplication by
 
x1

 . 
Ax = a1 · · · an  ..  = x1 a1 + · · · + xn an .
xn


To define matrix-matrix multiplication, let us suppose that A = a1 · · · an ∈


Rm×n and that B = b1 · · · bp ∈ Rn×p . Then the matrix-matrix product


C := AB is a matrix C ∈ Rm×p of the form A = c1 · · · cn ∈ Rm×p , where
each column satisfies

ci = A · bi .
In other words, we apply the matrix A to every column of B to obtain the
matrix C.
Example 1. Here are a few examples:
    
1 4 5 −1 1 13 11 −15
=
2 3 2 3 −4 16 7 −10
   
−1 −2   2 −3 −4
−1 5  1 3 2
= −6 −3 3 
−1 0 1
3 1 5 9 7
   
1
4 5 6
2 4 5 6 =  8 10 12
3 12 15 18
Recall that the matrix equation corresponds to the vector equation

x1 a1 + · · · + xn an = b.

Given two matrices A, of size m × n, and B, of size n × p, we see that AB, of

size m × p, satisfies
(AB) x = A (Bx) .

1
    
4 1 5 −1 1
Example 2. A = , B= . Then
3 2 2 3 −4
 
 x
5 −1 1  1 
      
5 −1 1
Bx = x2 = x1 + x2 + x3 .
2 3 −4 2 3 −4
x3
       
1 4 5 −1 1
A (Bx) = x1 + x2 + x3
2 3 2 3 −4
        
1 4 5 1 4 −1 1 4 1
= x1 + x2 + x3
2 3 2 2 3 3 2 3 −4
     
13 11 −15
= x1 + x2 + x3
16 7 −10
 
  x1
13 11 −15  
= x2
16 7 −10
x3
So, (AB) x = A (Bx) implies
           
13 11 −15 1 4 5 1 4 −1 1 4 1
AB = = , , .
16 7 −10 2 3 2 2 3 3 2 3 −4
Formally:

AB := Ab1 ··· Abp .
We can also define matrix-matrix multiplication via row by column multi-
plication:
(AB)ij = Rowi (A) · Columnj (B) = ai1 b1j + ai2 b2j + . . . + ain bnj .
Example 3. Here are a few examples how matrix-matrix multiplication behaves
very different
 fromthe multiplication
  of scalar
 numbers.
 Let
3 −6 −1 1 −3 −5
A= ,B= ,C= .
−1 2 3 4 2 1
Then we observe
    
3 −6 −1 1 −21 −21
1. AB = = ,
−1 2 3 4 7 7
    
−1 1 3 −6 −4 8
BA = = .
3 4 −1 2 5 −10
AB may not be equal to BA.
    
3 −6 −3 −5 −21 −21
2. AC = = .
−1 2 2 1 7 7
AB = AC does not necessarily imply B = C.
 
2 6
3. D = B − C =
 1 3   
3 −6 2 6 0 0
AD = =
−1 2 1 3 0 0
AD = 0 does not necessarily imply A = 0 or D = 0.

2
Matrix Transpose and Matrix Inverse
Definition 4. The transpose of an m × n matrix A is the n × m matrix AT
whose columns are the corresponding rows of A.
   
1 4 5 −1 1
Example 5. For A = ,B=
2 3 2 3 −4
 
T 1 2
A =
4 3
 
5 2
B T = −1 3 
1 −4
 
13 11 −15
AB =
16 7 −10
 
13 16
B T AT =  11 7  = (AB)T
−15 −10

What happens if we take the transpose of a product of two matrices? Notice

that if A ∈ Rm×n and B ∈ Rn×k , then AT ∈ Rn×m and B T ∈ Rk×n . So we have
AB ∈ Rm×k and AB ∈ Rk×m .
T
Theorem 6. (AB) = B T AT , provided AB is defined.

Proof. We have
 
T
(AB) = (AB)ji
ij
= (A)j1 (B)1i + (A)j2 (B)2i + . . . + (A)jn (B)ni
= (B)1i (A)j1 + (B)2i (A)j2 + . . . + (B)ni (A)jn
= (B)Ti1 (A)T1j + (B)Ti2 (A)T2j + . . . + (B)Tin (A)Tnj
= B T AT ij .

We have discussed the transpose of matrices. Next we discuss the inverse of

matrices.
Example 7. To Solve 5x = 3 we multiplied by 15 :

1 1
· 5x = · 3
5 5
3
x= .
5

3
 1
In general, to solve ax = b, we multiply by a, assuming that a 6= 0:

1 1
ax = b
a a
b
x= .
a
This was an example of a 1 × 1 system. What about general linear systems of
equations Ax = b?
Definition 8. An n × n matrix A is invertible if there is an n × n matrix A−1
such that AA−1 = A−1 A = I.
   
2 5 3 −5
Example 9. For A = , A−1 = . Indeed
1 3 −1 −2
    
2 5 3 −5 1 0
=
1 3 −1 −2 0 1
    
3 −5 2 5 1 0
=
−1 −2 1 3 0 1

Theorem 10. If A is an invertible n × n matrix, then Ax = b has the unique

solution x = A−1 b for every b ∈ Rn .
Proof.
1. Existence: A−1 b is a solution since (associativity)

A A−1 b = AA−1 b = Ib = b.

2. Uniqueness: If y is any solution; that is, Ay = b, then

y = A−1 Ay = A−1 b.

Example 11. Solve the system with matrix A from the previous example and
with right-hand side
 
1
b=
−1
The solution is
    
3 −5 1 −2
x= =
−1 −2 −1 1
Invertible matrices satisfy many important properties.
Theorem 12. Given n × n invertible matrices A, B.

4

−1
1. A−1 =A
−1
2. (AB) = B −1 A−1

−1
T
3. AT = A−1 ( AT is an n × n matrix whose columns are the corre-
sponding rows of A.)
Proof. We will only show one side, the other side is an exercise.

−1
1. AA−1 = A−1 A = I; thus A = A−1 .

2. B −1 A−1 (AB) = B −1 A−1 A B = B −1 B = I

T
3. We have seen that AT B T = (BA) . Hence, A−1 A = I implies

T

T
AT A−1 = A−1 A = I T = I.

So, how do find the inverse of an invertible matrix?

Theorem 13. Let A be an n × n matrix.
1. An n × n matrix A is invertible if and only if A is row equivalent to I.
2. The sequence of row operations that reduces an invertible n × n matrix A
to I also transform I to A−1 .
Example 14. We have
   
1 2 3 1 0 0 1 2 3 1 0 0
 2 3 5 0 1 0  −→  0 −1 −1 −2 1 0 
2 4 7 0 0 1 0 0 1 −2 0 1
 
1 2 3 1 0 0
−→  0 1 1 2 −1 0 
0 0 1 −2 0 1
 
1 2 3 1 0 0
−→  0 1 0 4 −1 −1 
0 0 1 −2 0 1
 
1 0 0 −1 2 −1
−→  0 1 0 4 −1 −1 
0 0 1 −2 0 1

Therefore  
−1 2 −1
A−1 = 4 −1 −1
−2 0 1
Here are a few more general criteria for checking whether a matrix is invert-
ible.

5
Theorem 15. Let A be an n × n matrix. Then the following statements are
equivalent.
1. A is an invertible matrix.
2. A is row equivalent to I.

3. A has n pivot position.

4. The equation Ax = 0 has only the trivial solution.
5. The columns of A form a linearly independent set.
6. The linear transformation x → Ax is one-to-one.

7. The equation Ax = b ha at least one solution for each b ∈ Rn .

8. The columns of A spans Rn .
9. The linear transformation x → Ax maps Rn onto Rn .

10. There is an n × n matrix C such that CA = I.

11. There is an n × n matrix D such that AD = I.
12. AT is an invertible matrix.