Beruflich Dokumente
Kultur Dokumente
MATHEMATICAL
PRELIMINARIES
Introduction to
Matrix Algebra
1-1.
DEFINITION OF A MATRIX
an
a,,,
a2 1 , a2 2 , a2n
am, am2, ,
amn
a 12
a21
a22
a,,1
am2
."
.'
a,,
a2
a]
(1-1)
amn
Note that the first term in the order pertains to the number of rows and the
second term to the nuniber of columns. For convenience, we refer to the order
of a matrix as simply m x n rather than of order m by n.
* In
CHAP. 1
A matrix having only one row is called a row matrix. Similarly, a matrix
having only one column is called a column matrix or column vector.* Braces
instead of brackets are commonly used to denote a column matrix and the
column subscript is eliminated. Also, the elements are arranged horizontally
instead of vertically, to save space. The various column-matrix notations are:
1-2.
MATRIX MULTIPLICATION
Two matrices, a and b, are equal if they are of the same order and if cor
responding elements are equal:
a= b
when
aij = bij
(1-3)
C1
JC2
;1c
== {Ci{= C
= c, _{C{C1C2,...,
(1-2)
If the number of rows and the number of columns are equal, the matrix is said
to be square. (Special types of square matrices are discussed in a later section.)
Finally, if all the elements are zero, the matrix is called a null matrix, and is
represented by 0 (boldface, as in the previous case).
- Z
~ALEEEIJII~
- xsm
SEC. 1-3.
a=b
i = 1, 2...,m
j = 1, 2,...,n
Addition and subtraction operations are defined only for matrices ofthe same
order. The sum of two m x n matrices, a and b, is defined to be the m x n
matrix [aij + bj]:
Similarly,
3 x 4 Matrix
4
3
2 -1
-7
4 -3
2
-
laij
For example, if
bij] = [
- bij]
b =o
0 -1J
then
(1-4)
(1-5)
-1
3
-11
01
I x 3 Row Matrix
[3 4 2]
atb=
a-b=
-2 -1 -1
and
3 x I Column Matrix
{4} or
4 or{3, 4, 2}
2 x 2 Square Matrix
a+b=b+a
(1-6)
a + (b + c) = (a + b) + c
(1-7)
[2 7
1-3.
2 x 2 Null Matrix
[o
L0
01
oj
MATRIX MULTIPLICATION
and
a =
then
a
-10
+ 35
CHAP. 1
(1-8)
2 x2
+ .-. + alnX
+ a 2 2 x2 +'''
a 2zxl
= C1
+ a2 nX,-: C2
SEC. 1-3.
MATRIX MULTIPLICATION
Xk =
k= 1, 2,...,n
bkjYj
j=1
(1-13)
+ a,,,nx,, = Cm
{ki=1
(j
bkjY)} =
i= 1,2,...,m
(ci}
(a)
k=l
akXk
i = 1, 2,...,m
= Ci
(a)
i= 1,2,...,m
Pij =
aikbkj
j= 1,2 ... ,s
k =1
where k is a dummy index. Using column matrix notation, (1-9) takes the form
t1
=i {ci}
aikXk
i =1,2,...,m
=1 iy
i= 1,2,...,m
j= 1,2,...,n
(1-11)
Since (1-10) and (1-11) must be equivalent, it follows that the definition
equation for a matrix multiplied by a column matrix is
py = c
(1-16)
where p is m x s and y is s x 1. Now, we also express (1-13), which defines
the transformation of variables, in matrix form,
x = by
ax = [aij] {Xi
i= 1,2....,m
aikxk}
=k1
(1-12)
This product is defined only when the column order of a is equal to the row
order of x. The result is a column matrix, the row order of which is equal to
that of a. In general, if a is of order r x s, and x of order s x 1, the product
ax is of order r x 1.
-
aby = c
(1-18)
and requiring (1-16) and (1-18) to be equivalent, results in the following
definition equation for the product, ab:
i== 1,2,..., m
-]
Pij =
a = 8 -4
0
3_
(1)(2) + (-1)(3)
3=:
]
1]
ax = (8)(2) + (-4)(3) =
(0)(2) + (3)(3) J
(1-17)
Example 1-2
(1-15)
(1-10)
(1-14)
k=l
k= ,2,...,n
j= 1,2,...,s
(1-19)
aikbkj
This product is defined only when the column order of a is equal to the row
order of b. In general, if a is of order r x n, and b of order n x q, the product
ab is of order r x q. The element at the ith row and jth column of the product
is obtained by multiplying corresponding elements in the ith row of the first
matrix and the jth column of the second matrix.
CHAP. 1
SEC. 1-4
a = [a/']:
all
Example 1-3
a= -1 i
0 2
(1)(1) + (0)(0)
b= I0
(1)(1) + (0)(1)
TRANSPOSE OF A MATRIX
a =[aij
]
0 -11
3]
1 -1
1
'aln.
a2n
a aai
21
a22
Lai
alm2
-al
a2 l
...
a1 2
al2 2
Lain
a2
(1-23)
(m x n)
)
()( -1) + (0)(3)j
1) (- 1)(- 1) + (1)(3)
()(- ) + (2)(3) 3
(0)(0) + (2)(- 1)
aT=
amn
am
(n x m)
. . amn
The element, aiT, at the ith row and jth column of aT , where now i varies from 1
to n and j from 1 to m, is given by
T
(1-24)
a j = ai
a(b + c) = ab + ac
(b + c)a = ba + ca
(1-22)
ab : ba
La 21
a2 2 jLb2i
bll
b12 [a1
b22 J La2l
Lb2 1
b;1
b22J
alb 1
La2 lbll
a,2 1
Fbllall + b 2 a2 l
a2 2 J
a22b21
aazb22
]7
a=
a T1=
(1-21)
ra11
where aji is the element at the jth row and ith column of a. For example,
,an] =
[al, a2 ,
li}T
(1-25)
We consider next the transpose matrix associated with the product of two
matrices. Let
p = ab
(a)
where a is m x n and b is n x s. The product, p, is m x s and the element,
Pu, is
n
i = 1,2,...,m
(b)
Pij = E aikbkj
j= 1,2,...,s
k=l
The transpose of p will be of order s x m and the typical element is
a 21b12 + a22b22
Py = Pji
bl 1 a 12 + bl 2 a 2 2
b21 a 12 + b22 a 22 j
where now i = 1, 2,...,s and j = 1, 2,...,in. Using (1-24) and (b), we can
write (c) as
PT =
k=l
(c)
ajkbki =
bkakj
k=1
= 1,2,..., s
j = 1,2,...,m
(d)
TRANSPOSE OF A MATRIX
pT =
(ab)T =
bTaT
(1-26)
Equation (1-26) states that the transpose of a product is the product of the
CHAP. 1
10
SEC. 1-5.
11
(1-28)
i =j
ciE5.= +1
Example 1-4
I = [abij]
-5
d = [dibij]
4-
ab= 13}
(1-29)
b ={}
a= 7 1
i,j = 1, 2, . . , n
(ab) T = [4
(1-30)
where dj, d2, ... , d are the principal elements. If the principal diagonal elements
are all equal to k, the matrix reduces to
13 6]
[kbij] = k[lij] = kI
Alternatively,
aT =
(ab)T = bla T
3 1
=[2
b =[2
-1]
] = [4
-1L2
3 6]
(1-31)
(I -2)
Ima = a
1-5.
If the numbers of rows and of columns are equal, the matrix is said to be square
and of order n, where n is the number of rows. The elements aii (i = 1, 2 .. , n)
lie on the principal diagonal. If all the elements except the principal-diagonal
elements are zero, the matrix is called a diagonal matrix. We will use d for
diagonal matrices. If the elements of a diagonal matrix are all unity, the diagonal
matrix is referred to as a unit matrix. A unit matrix is usually indicated by I,,
where n is the order of the matrix.
A unit matrix is commutative with any square matrix of the same order.
Similarly, two diagonal matrices of order are commutative and the product
is a diagonal matrix of order n. Premultiplication of a by a conformable
diagonal matrix d multiplies the ith row of a by dli and postmultiplication
multiplies the jth column by dj.
rllo
v --
r_
r2
_0]
][2
Example 1-5
1-I
[3 2]
Diagonal Matrix, Order 3
_0
0] 2
0 [3 1]
2
2 72
= 6
2[6
---
_7
0 70][ -1]=[4 -7
A square matrix a for which ai = aj, is called symmetrical and has the
property that a = aT. If ai = -aji (i # j) and the principal diagonal elements
all equal zero, the matrix is said to be skew-symmetrical. In this case, a = -a.
Any square matrix can be reduced to the sum of a symmetrical matrix and a
skew-symmetrical matrix:
a = b+ c
0 0
050
003j
12:[I 0]
I2=
__
bij =
( a ij +
Cij =
(aij
ai)
i)
(1-33)
1r2
CHAP. 1
The product of two symmetrical matrices is symmetrical only when the matrices
are commutative.* Finally, one can easily show that products of the type
(aTa)
(aTba)
(aaT)
SEC. 1-6.
Example 1-7
where
1-6.
1[ablbJ
L 21 b1
b2 2 j
a2 b
0
a2b2
2
21
22
Onerations. on
alt
a = a2
a31
a1t2 a,
a2 2
a32
a2 3
a3 3
a,, aI2 1 a1 3
= 2 a a2 2
a2 3
a3
a3 2
a3 3
all a1 2
a13
a2 l
a23
31
a2 2
a32
a3l
a23
, a33j
or
[A
or _
a=
A_
A12
A21
A
2221
2
A1
13
La2 3
A
A22 = [a33]
a32]
IB 11 B11 2
b LB2 1 B2 2
A22 j
where Bj and A,, are of the same order. The sum is
a = All A12
LA2
(1-34)
All + 31
a+b
1+
B21 I
A1 2 +
2
A2 2 + B22
(1-35)
a1 3 3
Note that the partition lines are always straight and extend across the entire
matrix. To reduce the amount of writing, the submatrices are represented by
* See Prob. 1-7.
a32
al1 3
If two matrices of the same order are identically partitioned, the rules of
matrix addition are applicable to the submatrices. Let
a3
:all a12
A21 =
1bl
a22
la23 1 a 2
-3 0 0
7
12 4
a12
al
A2]
1
A
A22
12 A,2
k=Cat
Lower TriangularMatrix
a2
fall
a
a = FAllt
3 5 2
07
f
0 4
a2 l
We represent
as
Upper TriangularMatrix
13
a single symbol. We will use upper case letters to denote the submatrices
whenever possible and omit the partition lines.
all
=1,2...N
j= 1, 2,...,M
,/ = 1,2.
M
k= 1,2,...,S
(1-36)
c = ab = [Cik]
M
Cik = E AijBk
j=1
i= 1,2,...,N
k= 1,2,...,S
- (1-37)
when the row partitions of b are consistent with the column partitions of a.
14
CHAP. 1
all a
=a21
a31
b,
a22
a2
b2
a3 2
a133
(b3J
ab = a[B,,B, 2] = [aB 1,
[A1 1
A12
A1 ,
A2
A2 2
...
A2 n
Am
Arn2
[AT
A21
then
a3 3
a =
B2
{B J= A1 B1 , +
A1 2 B21
a2,
a22
a31
a3 2 a3
a23
b,
al,
a-
,2 a,3
a21
a22
a2 3
La3
a32
a3 3
Ai11
I A2 l
LA21
b=
Allb
A21b
b= b2,1
b31
A;2
a=
a22
a2 3
a3
a3 3
a=
[Ai 1 J]
where
Al = [a,,]
A2 =
22 a
a32 a23
33
and 0 denotes a null matrix. The product of two quasi-diagonal matrices of
like structure (corresponding diagonal submatrices are of the same order) is
a quasi-diagonal matrix of the same structure.
Since the column order of A11 and A21 is equal to the row order of b, no
partitioning of b is required. The product is
ab = IAll
...
I b22
b3l b32
Arm
ATr 2AT
Aln
b2
n.n
(1-38)
' aB 12]
Suppose we partition a with a vertical partition between the second and third
columns.
all a12
a 13
a = a21 a22
a2 3 = [A 11A1 2 ]
a31 a3 2
15
SEC. 1-6.
A,
OB
A2 ..
00
B2
AB,
A2B2
*'''* 0O
* '".0
- O
(1-39)
.. A
...-
B,
0o
'"
AnBn
]
where Ai and Bi are of the same order.
We use the term quasi to distinguish between partitioned and unpartitioned
matrices having the sameform. For example, we call
LA
11 0A2
b22 = [B11B12]
b32
In this case, since the row order of B,, and B12 is the same as the column
O 1
.0' O
A21
a lower quasi-triangular matrix.
A22
(1-40)
16
CHAP. 1
SEC. 1-7.
1-7.
(a)
a2 1 x 1 + a2 2 X2 = C2
Integer
3
1
4
2
2 1)x 1
= Cia
- C2 a 2
22
+ c2 all
(b)
al 1
a12
a2 2
-lal
= aa
22
-al2a2
(1-41)
We use the terms array and matrix interchangeably, since they are synony
mous. Also, we refer to the determinant of an nth-order array as an nth-order
determinant. It should be noted that determinants are associated only with
square arrays, that is, with square matrices.
The determinant of a third-order array is defined as
all
a1 2
a1 3
+a1 la 2 2 a3 3 - aa
a21
a2 2
a2 3 =
-al
a31
a32
a3 3
+a1
32a 1 a
23
a
2
2 3 a3 2
a33 + at 2 a2 3 a31
(1-42)
a1 3 a2 2a 3 1
22
Inversions
(3, 1)(3, 2)
None
(4, 2)
None
Total
2
0
1
3
17
2 3a 3 2 ,
These properties are associated with the arrangement of the column subscripts
and can be conveniently described using the concept of a permutation, which
is discussed below.
A set of distinct integers is considered to be in natural order if each integer
is followed only by larger integers. A rearrangement of the natural order is
called a permutation of the set. For example, (1, 3, 5) is in natural order and
This set has three inversions. A permutation is classified as even (odd) if the
total number of inversions for the set is an even (odd) integer. According to
this convention, (1, 2, 3) and (3, 1, 2) are even permutations and (1, 3, 2) is an
odd permutation. Instead of counting the inversions, we can determine the
number of integer interchanges required to rearrange the set in its natural order
since an even (odd) number of interchanges corresponds to an even (odd)
number of inversions. For example, (3, 2, 1)has three inversions and requires
one interchange. Working with interchanges rather than inversions is practical
only when the set is small.
Referring back to (1-41) and (1-42), we see that each product is a permutation
of the set of column subscripts and the sign is negative when the permutation
is odd. The number of products is equal to the number of possible permutations
of the column subscripts that can be formed. One can easily show that there
are n-factorial* possible permutations for a set of n distinct integers.
We let (al, a 2 ... a, ) be a permutation of the set (1, 2 .. , n) and define
el2
.. a
as
e 12 ... , = + I
e,-...~,,
O=
-1
. . .,
(1-43)
a12
a21
a2 2 ''
-..
(1-44)
an
a2
...
where the summation is taken over all possible permutations of (1, 2, .. , n).
* Factorial n = n! = n(n -
)(n - 2) . .(2)(1).
18
CHAP. 1
SEC. 1-8.
19
Example 1-8
The permutations for n = 3 are
a2 2
La11
al = 1
a2 = 2
C3 = 3
C
=
1
a2 =
al =
X2 =
al = 2
a, = 3
al
a,2 1
= 3
2
= 3
C3 =
e13 2 = -1
e2 1 3
C3 = 1
= -1
e2 3 1 = +1
= 3
e12 3 = +1
= I
C2 =
e3 12 = +1
X3 =
e3 2 1 = -I
C
2 =
a12j
la'l = a2,a 1 2 - aj
a2 2 = -a
1
Property 4 is also obvious from (b). To demonstrate the fifth, we take
a21
Then
Next, let
kall
a12
a13
a21
a2
a23
a3 l
a3 2
a33
alla22a3 3 - aa
= -a2a2a33
+a13a21a32
23
a32
1. If all elements of any row (or column) are zero, the determinant is zero.
2. The value of the determinant is unchanged if the rows and columns are
interchanged; that is, aT = al.
3. If two successive rows (or two successive columns) are interchanged, the
sign of the determinant is changed.
4. If all elements of one row (or one column) are multiplied by a number k,
the determinant is multiplied by k.
5. If corresponding elements of two rows (or two columns) are equal or in
a constant ratio, then the determinant is zero.
6. If each element in one row (or one column) is expressed as the sum of
two terms, then the determinant is equal to the sum of two determinants,
in each of which one of the two terms is deleted in each element of that
row (or column).
7. If to the elements of any row (column) are added k times the cor
responding elements of any other row (column), the determinant is
unchanged.
We demonstrate these properties for the case of a second-order matrix. Let
a =
at 2 = b1 2 + c12
where
12
According to property 6,
a2a23a3
-'a3a22a31
ka
all
a2 2
Jbl = b1,
bl 2
a21
a2 2
Il = c1 1
a21
c122
a2 2
This result can be obtained by substituting for a,,ll and a1 2 in (b). Finally, to
illustrate property 7, we take
bl = a,, + ka 21
bl 2 = a
b2 l =
+ ka 2 2
b22 = a2 1
Then,
tb = (a,, + ka 2 )a22
1-8.
a21
-(a 12 + ka 2 2 )a2l =
fat
If the row and column containing an element, aij, in the square matrix, a,
are deleted, the determinant of the remaining square array is called the minor
of aij, and is denoted by M. The cofactor of aij, denoted by A, is related to
the minor of Mij by
Aj
As an illustration, we take
a=
at
Xal 2
a2 1 a22
(-
1)i'+jMi
1 7 4
5 3 1
The determinant is
la = alla22 - al2a2
Properties 1 and 2 are obvious. It follows from property 2 that aTi = al. We
*See Probs. 1-17, 1-18, 1-19.
53=
-1
3 81
1= -37
:5=
A 2 3 = (- 1) 5M 23 = + 1
A2 2 = (--1) 4 M 2 2 = -37
(1-45)
20
CHAP. 1
|al =
k=
aikAik =
(1-46)
akjAkj
k=l
Equation (1-46) states that the determinant is equal to the sum of the products
of the elements of any single row or column by their cofactors.
Since the determinant is zero if two rows or columns are identical, if follows
that
r
E
arkAik = 0
k=l
CRAMER'S RULE
21
Ic = al bl
(1-47)
Y akAkj = 0
k=l
SEC. 1-9.
Whether we use (1-48) or first multiply a and band then determine ab depends
on the form and order of a and b. If they are diagonal or triangular, (1-48)
is quite efficient. t
The above identities are used to establish Cramer's rule in the following section.
Example 1-10
Example 1-9
(1) We apply (1-46) to a third-order array and expand with respect to the first row:
all
a1 2
a1 3
a2 1
a2 2
a2 3
a3 1
a3 2
a3 3
(1)
3al:5 b=
lal = -4
|b = 5
40
|c = -20
Alternatively,
= a (_1)
1
a2 2
a2 3
l132
a3 3
+ a2(-1)3
= al(a22a33 a2 3 a3 2) + a
2 (-a 2
a21
a2 3
a3 1
a33
ta 3 3
+ a 1 3 (-
)4
21
a31
c =[5
c= l1
a2 2
a3 2
+ a2 3 a31 ) + a1 3 (a21 a3 2 - a2 2 a3 1 )
15
29
(2)
= [
To illustrate (1-47), we take the cofactors for the first row and the elements of the second
row:
lal =5
3
A a2 kAlk
clJ= -20
and
b=
Ibl= 8
4]
cl = +40
k=l
= a21(a22a33 - a23a32 )
a22(--a21a33 -
a23a3
+ a23(a2a32 - a22a31) =
c=
a11
a21
a2 2
a32
a3 3
a3l
a,,
(22
a3 2
a33
= (a1 1 )(a22 a3 3 ) =
110a22a33
Generalizing this result, we find that the determinant of a triangular matrix is equal to
the product of the diagonal elements. This result is quite useful.
1-9.
20
and
CI= +40
CRAMER'S RULE
k=l
12]
5
(2) Suppose the array is triangular in form, for example, lower triangular. Expanding
with respect to the first row, we have
ajkXk = Cjj
= 1, 2,...,
(a)
22
CHAP. 1
k=
k=l
j=l
j=l
Ajrj
E Ajrcj
j=1
(c)
j=l1
Note that the inverse matrix is defined only for a nonsingular square matrix.
Example 1-11
a= 2
4 1 2
-10
+7
-l
-- I
-7
-10
+5
a1= --Adj a =
al
+7 -1
--1/5 + 1/25
O +2/5
+2/5
-7/25
/251
-1/5
+1/25J
(a)
can be expressed as
n
i=1,2,...,n
(1-50)
-10
inverse a = a - = . Adj a
lai
Adj a = [Aij]T
Xi= Ajici
tatj1=
(1-49)
i,j= 1,2,...,n
[Aij]= -1 -o10
7 +5
23
We define the adjoint and inverse matrices for the square matrix a of order n as
(d)
only in that the rth column of a is replaced by c. Equation (c) leads to Cramer's
rule, which can be stated as follows:
1-10.
The expansion on the right side of (c) differs from the expansion
la = Z a,.Aj.
~1-10.
(b)
Now, the inner sum vanishes when r k and equals tat when r = k. This
follows from (1-47). Then, (b) reduces to
alx r =
SEC.
(b)
(note that we have taken r = i in Eq. c of Sec. 1-9). Using matrix notation,
(b) takes the form
I
xi=- [Ai~l {cj}
(c)
Equation (c) leads naturally to the definition of adjoint and inverse matrices.
X = a-lc
(d)
Substituting for x in (a) and c in (d), we see that a- 1 has the property that
ala
= aa-I
(1--51)
Equation (1-51) is frequently taken as the definition of the inverse matrix
instead of (1-50). Applying (1-48) to (1-51), we obtain
la-' lal = I
i
It follows that (1-51) is valid only when tlat # 0. Multiplication by the inverse
matrix is analogous to division in ordinary algebra.
If a is symmetrical,. then a- is also symmetrical. To show this, we take the
transpose of (1-51), and use the fact that a - aT:
(a- a)r = aa-
r = In
24
CHAP. 1
SEC. 1-11.
Premultiplication by a- results in
. b',
r
=
,-
(1-52T
2)
\- E_,
1 0 0
interchanges rows 1 and 3 and postmultiplication by
We consider next the inverse matrix associated with the product of two square
matrices. Let
c = ab
where a and b are both of order n x n and nonsingular. Premultiplication
by a-' and then b- results in
a-'c = b
(b-'a-')c = I
(1-53)
1 0
and therefore a-' is also symmetrical. One can also show* that, for any
nonsingular square matrix, the inverse and transpose operations can be inter
changed:
25
1 0 00
0
0 1
0100
rs:;mo
-I}
__
1 1/2 1/5
a=
-2
We first:
1. Add (-3) times the first row to the second row.
2. Add (2)times the first row to the third row.
* See properties of determinants (page 18).
CHMAP. I
26
Step 3
0
2/11
-6/11
+ 34/11
-3 1 0
2 0 1
and the result is
0 11/2 7/5
27/5
0 2
01
1/51
1/2
\1
1/2 1/5
o (
3
-2
0
1
1-12.
Finally, we multiply the third row by 55/269. The complete set of operations is
o1t
0
55/269
1 FI
2/11 0 -3
01
0
1
L0
-2
1
0
2
0111
1
l2j
3 7
-2 1
1/5
2
5
1/2 1/5
=-_
1
0
14/55 =b
1
This example illustrates the reduction of a square matrix to a triangular matrix using
elementary operations on rows, and is the basis for the Gauss elimination soluion scheme
(Refs. 9, 11, 13). We write the result as
ea== b
where e is the product of the four operation matrices listed above:
0
0
1
e=
6/11
L+ 1870/2959
55/2691
1/2
1/5
7
1
2
5
and
1/5
14/55
1
1
0
(1-54)
RANK OF A MATRIX
The rank, r, of a matrix is defined as the order of the largest square array,
formed by deleting certain rows and columns, which has a nonvanishing deter
minant. The concept of rank is quite important since, as we shall see in the
next section, the solvability of a set of linear algebraic equations is dependent
on the rank of certain matrices associated with the set.
Let a be of order mnx n. Suppose the rank of a is r. Then a has r rows
which are linearly independent, that is, which contain a nonvanishing deter
minant of order r, and the remaining n - r rows are linear combinations of
these r rows. Also, it has n - r columns which are linear combinations of r
linearly independent columns.
To establish this result, we suppose the determinant associated with the first r
rows and columns does not vanish. If a is of rank r, one can always rearrange
the rows and columns such that this condition is satisfied. We consider the
(r + I)th-order determinant associated with the first r rows and columns, row
m r < q < In.
p, and column q where ri <p in,
2/11
- 220/2959
1 1/2
269/55
0 0
27/5
0 2
1870/2959
01
where p and q are nonsingular. This follows from the fact that the elementary
1
0
0 -2
b = paq
Next, we add (-2) times the second row to the third row:
Step 4
0
2/11
-220/2959
14/55
27/51
1
11/2 7/5 = 0
2
27/5_1
1 0 2
2'11
- 4/11
0J[ 1
-6/11
0
Two matrices are said to be equivalent if one can be derived from the other
by any finite number of elementary operations. Referring to Example 1-12, the
matrices
1/51
1 1/2
27
RANK OF A MATRIX
SEC. 1-12,
a1 1
a2
55/269
We obtain e by applying successive operations, starting with a unit matrix. This is more
convenient than listing and then multiplying the operation matrices for the various steps.
The form of e after each step is listed below:
Step 2
Step 1
Initial
01O
0
1
0 01
[
-1 0 0
- 6 /11 2/ll 0
-3-1 0
0 1 0
1
0
2
2 0 1
0 0 1
Ar |,Ii
at 2
a2 2
air
a.
a2q
(1-55)
- apl
ar2
ar
ap 2
arq
'''arr
ap
We multiply the elements in rowj by 2pj (j = 1, 2,...,r) and subtract the result
from the last row. This operation will not change the magnitude of A,r+ (see
Sec. 1-7). In particular, we determine the constants such that the first r elements
28
CHAP. 1
We see that a is at least of rank 2 since the determinant associated with the first two rows
and columns is finite. Then, the first two rows are linearly independent. We consider the
columns 1,2, and q:
array consisting
determinant of the third-order
I~~~~ of a,
I
a2 1
.112
a22
air
,
. I ar
\ 5p r
a,r2
P2
''
ap,)
=
_=
h....,n 1-
_.^
slngulal.
(1-56)
p2
alq
A3 = 2
a2q
(13q
a2r
matrix is non-
re.Cduces to
1nlh t
-J
a, 2
a2 l
a2 2
arn
ar2
we obtain
-2qa
-a2r
a,-
A2 = 1
2, = 3
(1-57)
Ar+
221 + 22 = 7
a,,. ialq
al1
29
RANK OF A MATRIX
SEC. 1-12.
C6tIq
a3q = ,lalq
q= 3,4
where
aa0)- [a
a
pq
aq
aa
pq
(1-58)
rq
A,+ vanishes
Applying Laplace's expansion formula to (1-57), we see that
when a(") = 0.
ofp and q. It
Now, if a is of rank r, A,+ 1 vanishes for all combinations
follows that
q = r + 1, .. .,n
p = r + 1, r + 2,...,in
(1-59)
a n,, a2,n
One can show* that the elementary operations do not change the rank of
a matrix. This fact can be used to determine the rank of a matrix. Suppose b
defined by (1-61) is obtained by applying elementary operations to a. We know
that b and a have the same rank. It follows that a is of rank p. A matrix having
the form of b is called an echelon matrix. When a is large, it is more efficient
to reduce it to an echelon matrix rather than try to find the largest nonvanishing
determinant:
al
a1 2
6121
a2 2
...
ain
.C12n
ra
(-apl60)
C[at
am2
b12
...
-0 0
(011 - p) x p)
arn
pr j
Example 1-14
1 2
of the first 1- rows. One can also show* that the last n - r columns of a are
linear combinations of the first r columns.
Example 1-13
Consider the 3 x 4 matrix
a= 2 1 3 2
5 7 12 12
34
a =j2
5s 7
t2
(p x (n--p))
4
2
3
0
t4
*See Prob. 1-40.
blp
bp21
B12
(1-61)
.- 1 _
aln
Equation (1-60) states that the last min- r rows of a are linear combinations
I
b
= b-
aII,
r2
(P X )
ipr
a.1. a.2
a1 2 a 2
Since a33 and a34 satisfy this requirement, we conclude that a is of rank 2. The rows are
related by
-3
-3
-6
-8
I ((-
p) x (n-p))
30
CHAP. 1
SEC. 1-13.
2
3
4
-3 -3
-6
0
0 2
At this point, we see that r = 3.To obtain b,we multiply the second row by
- 1/3, the third
row by - 1/2, and interchange the third and fourth columns:
= [alia
a22
(1-62)
One can show* that the rank of a cannot be greater than the minimum value
of r associated with b and c:
r(a) < min [r(b), r(c)]
(1-63)
a= [1/2
(1-66)
{X I
AIX =
(nt x n) (n x s)
A2]
I 1a23
XI
1 2 1
0 1 0
a2 331 = [Ai
a
k21
3={2}
b=
31
01[
a=[0 1
It follows that a is of rank 1.
A2X2
(1-67)
Since JIAI
0, it follows from Cramer's rule that (1-67) has a unique
solution
for X1. Finally, we can write the solution as
X = A '( - A,X,)
(1-68)
Since X2 is arbitrary, the system does not have a unique solution for a given c.
The order of X2 is generally called the defect of the system. The defect for this
system is 1.
If a is of rank 1, the second row is a scalar multiple, say , of
the first row.
Multiplying the second equation in (1-64) by 1/,, we have
allX
a, X
+ a12X2 + a13 x 3
= C,
(1-69)
If c2 Ac,
1 , the equations are inconsistent and no solution exists. Then, when
a is of rank 1,(1-64) has a solution only if the rows of c are related
in the same
manner as the rows ofa. If this condition is satisfied, the two equations
in (1-69)
are identical and one can be disregarded. Assuming that a
0, the solution is
1
1-13.
X = Olai)(C
a2
al3x
a 21
a22
a23
CIt
c2
(1-64)
ax = c
Suppose a is of rank 2 and
al
a2 t
* See Prob. 1-44.
a0 21 # 0
a22
(1-65)
aj2X2 - a3X3)
(1-70)
The defect of this system is 2.
The procedure followed for the simple case of 2 equations in
3 unknowns is
also applicable to the general case of m equations in n
unknowns:
Fail
a12
21
a22
a,,
a,,
C
X
C2
J Cn
(1-71)
If a is of rank in, there exists an mth order array which has a nonvanishing
determinant. We rearrange the columns such that the first m
columns are
32
REFERENCES
CHAP. 1
a 2l
a1 2
aim
a 22
.'
a2,n
al, m+
aln
a2 m+l '*'
all
a2n=
(xn m)
am 1
am2
*'
{XI
X2
am,,
'
am, m +
X
...
Cn+i1
A2 ]
..
(1-72)
{
Xn}
Xl
(mx 1)
x2
((n-m)x 1)
AIX 1 = c - A2X 2
(1-73)
"
I_ 4i
--
1 + a 12 X2 + a1 3X 3 =
C1
(a)
a2 1 X1 + a2 2 X2 + a x = C
23 3
2
a3 1 x 1 + a3 2 X2 + a X = C
33 3
3
am
a1 2
al,,
..
a2 2
21
(m x (n- m))
amn
we write (I-71) as
Since AlI I
33
In general, (1 -71) can be solved when r < m if the relations between the rows
of a and c are identical. We define the augmented matrix, a, for (1-71) as
'
.
a,,m2
C2
a 2,,
.
an
= [a
c]
(1-74)
When the rows of a and c are related in the same way, the rank of CXis equal
to
the rank of a. It follows that (1-71) has a solution only if the rank of the aug-.
mented matrix is equal to the rank of the coefficient matrix:
r(ot) = r(a)
(1-75)
Note that (1-75) is always satisfied when r(a) = m for arbitraryc.
We can determine r(a) and r(a) simultaneously using elementary operations
on provided that we do not interchange the elements in the last column.
The
reduction can be represented as
a=[a
C]
| AI
L
I C)
(1-76)
where A is of rank r(a). If C ) has a nonvanishing element, r(a) > r(a) and
no solution exists.
When r(a) = r(a), (1-71) contains r independent equations involving n un
knowns. The remaining m r equations are linear combinations of these
r equations and can be disregarded. Thus, the problem reduces to first finding
r(a) and then solving a set of I- independent equations in n unknowns.
The
complete problem can be efficiently handled by using the Gauss elimination
procedure (Refs. 9, 11, 13).
IC 1 +
1lC 1
(c)
C2
To show this, we multiply the first equation in (a) by -2/, the second by these equations the third equation. Using (b), we obtain
0 = C3 --
(b)
2,
and add to
- A2C 2
(d)
Unless the right-hand side vanishes, the equations are contradictory or inconsistent and no
solution exists. When c = 0, (c) is identically satisfied and we see
that (a) has a nontrivial
solution (x # 0) only when r < 3. The general case is handled in the same manner.*
REFERENCES
1. FRAZER, R. A., W. J. DUNCAN and A. R. COLLAR: Elementary
Matrices, Cambridge
University Press, London, 1963.
2. THOMAS,
3.
4.
5.
6.
7.
8.
1952.
9. Faddeeva, V. N.: ComputationalMethods of Linear Algebra, Dover
Publications,
New York, 1959.
10.
11.
12.
13.
(d)
[a 1
1-3.
C2 j
a22J
35
PROBLEMS
CHAP. 1
34
E ak
S 1 = al + a2 + a3 =
k=1
14. VARGA, R. S.: Matrix Iterative Analysis, Prentice-Hall, New York, 1962.
15. CONTE, S. D.: Elementary Numerical Analysis, McGraw-Hill, New York, 1965.
S2 = bl + b
S 1S 2
k=1
can be written as
PROBLEMS
1-1.
bk
b3 =
akbj
k=l j=l
(a)
db_ db k
d=Ldaik
Ldv
dy
3 2 1 + 7 1 3
0 5 6
5 1 0
l 6dy
dy
(b)
d7
E
1-2.)
*5
c = ab
51
31
fli4
2J
-1
then
db
dc
-= a-+
dy
(e
11
21
dy
da
-dy
1-5. Consider the triple product, abc. When is this product defined? Let
p = abe
Determine an expression for Pij. What is the order of p? Determine Pii for the
case where c = a T.
1-6. Evaluate the following products:
(a)
[2
-3]
(b)
(a + b)(a + b)
b,,
b,,}
(a)
(b)
aTa
{al, a2, .. , an} [b, b2, . ,
(c)
Kc
p0
'a,
C2
a2 1
a1 21
a2 2]
,]
(b)
aarba
Tba
where b is symmetrical
bTa Tcab
where c is symmetrical
(c)
qr-
indicated sub
1-9. Evaluate the following matrix product, using the
2
3
matrices:
1_13
3
2 [1
5 1
t '~ 2
|
ab. Show that the horizontal partitions of c correspond to
=
1-10. Let c
of b. Hint: See
those of a and the vertical partitions of c correspond to those
Eq. (1-37).
locations of
1-11. A matrix is said to be symmetrically partitioned if the
example,
For
the row and column partitions coincide.
all al 2 1 a31
a 21 a22 I .....
a23
.......--
a3
a3 3
a3 2
and
is symmetrically partitioned
aal1
__
a31
__--
2
a32
---
All
a
A22
A21
are placed
(b) Suppose we symmetrically partition c. What restrictions
partition
also
must
we
that
follow
it
Does
b?
and
a
of
on the partitions
1-10.
Prob.
See
Hint:
symmetrically?
b
a and
1-13. Consider the triple product,
c = aTba
n. Suppose
where b is a symmetric rth-order square matrix and a is of order r x
indicated
are
matrices
partitioned
the
we symmetrically partition c. The order of
(pxq)
(pxp)
in parentheses.
C 12 ]
(nxn)_ C11
l C2
(qxp)
(r x q)
A2 ]
a =[ A
(b) Express Cjk (j, k = 1, 2) in terms of Al, A2, and b.
1-14. Let d = [Dj] be a quasi-diagonal matrix. Show that
da = [DjAjk ]
bd = [BjkDk]
when the matrices are conformably partitioned.
1-15. Determine the number of inversions and interchanges for the
following sets.
(a)
(b)
(4, 3, 1, 2)
(3,4,2, 1)
How many permutations does (1, 2, 3, 4, 5)have?
Consider the terms
eala 2
ala
a2
(a)
a3a3
(b)
The first subscripts in (a) are in natural order. We obtain (b) by rearranging (a)
such that the second subscripts are in natural order. For example, rearranging
eflf 2fl3 fl, laf 2 2af 33
a33
(rxp)
(r x n)
and
23
(a)
of c.
Show that the following partitioning of a is consistent with that
1-16.
1-17.
a 13
i2
a2 2
a2
37
PROBLEMS
CHAP. 1
C2 2 1
(q x q)
we obtain
= (2, 3, 1)
8 = (3, 1, 2)
Show that if (a1, x2, a 3) is an even permutation, (l, f2, ,3) is also an even
permutation. Using this result, show that
e312 a3 a2
a23
exlx2-3alaa2a32.3 =
E
e#Pl3 apa2all22al,3
and, in general,
lal =afl
1-18.
Suppose that
be
a2l
ela2a3aloa2,2 a323
(a)
exlb 1 blalb2.ab3a3
(b)
b2a2 =
al,,a2
b33 =
a3a3
Show that
(c) = -(a)
that the sign of a determinant is reversed
establish
and
Generalize this result
when two rows are interchanged.
1-19. Consider the third-order determinant
la =
e,,
23lalla2.2a33
38
CHAP. 1
Show that I
= 0. (Hint: e,,,,,
kclj
I
this result and establish
1 23
1 2 3
(_ 1)In-pl
0 0 5 3
Generalize for
d = A, 6,jl
Let
(pxq)
g= Gil G
(G2q
G22 _
(q X p)
(q X q
Show that
0 01
3 0 0
0 2 1
(pxp)
d -2
1-24.
39
show that dl =
= -e,l,23). Generalize
3 1 2
2 3 1 -
PROBLEMS
PROBLEMS
i,j,k = 1,2, 3
(b)
1ejk
a11
a12
72 1
a2 2
''.
a1,1
q11
...
a2
g2
9222
2
''
bl,,b 2
b2 2
E ali(i=l
1)i+Mli
(c)
aiiAli
i=l
0 '-
...
...
bll
b,2
...
blp
al
b2 l
b2 2
...
b2p
a 2 1 a22
bn
b,,2
b P
anI
an
J L,1
J
a2
...
a12 ...
...
aan2
d=
(q x p)
(qxq)
OlrIp
(pxp)
(pxq)
A121
0[B,,
A21
A2 2
G2 1 G 2 2 L O
(qxq)
(qXp)
a2,
nn
(qxq)
(pxp)
(pxq)
B12
(qxp)
RB
22 j
(qxq)
G2 1B12
GllBll = All
GliB12 = A12
G2lBl = A21
G 2 2 B2 2 = A 2 2
IA,11 =
and
IG,,
1 1B,,I
By expressing d as
d = [
0O
(pxq)
0
D2I
D'
(pxn)
ann
D1
0
(p x q)
All
1---- = lal
aln = ---(nxp)
(nxn)
a
(pxp)
(qxp)
0
(pxp)
../.nn
b2n
We introduce symmetrical partitions after row (and column) p and write the
product as
O0
0
az.
,,,,
b
...
I[eIq D2j
Igf 0
IbN0
CHAP. 1
determine the four matrix equations relating Bjk and Ajk (j, k = 1, 2). Use this
result to find the inverse of
aj
a2
jl aj
j = 1,2,. ..
a ii
-'-
41
PROBLEMS
t esna:
40c
The determinant of the array contained in the first j rows and columns is called
the jth-order discriminant.
1-26. Does the following set of equations have a unique solution?
1 2 4
2 1 2
1-31. 'Find the inverse of
01
F,(tq
Note that A is (p x q).
1-32. Find the inverse of
1-27.
x1
2} =
D2
1-33. Use the results of Probs. 1-31 and 1-32 to find the inverse of
FLO
b=
3
7
1 0
d-r:
1 B12 1
B2 2
L
i1
13
I[
LO
1
[1 3
13
5 1
2
1
1-35.
Let
(d)
(pX p)
(px q)
All
A1 2
LA21
(q
Ia12a,3
ill
131 l3
AL,,
2,
Al22 j
A,
CL1[a,2 ia33
and
cJ
OlAt'
o I
l-A21 IqjL
C = (A2 2 -A
a
"I
pB,
- LBI
p)
A
2 2 J
(qxq)
where A1,1 # 0 and ja1 0. Show that the following elementary operations
on the partitioned rows of a reduce a to a triangular matrix. Determine A(U.
Let
a=
a,22 Ii
,,1 a,
aI
2
1 2
1 3
1 2
a=
21 Lt
B 22
3]
21
(b)
(c)
Lo
B,221
1 322j
where the order of Bjk is the same as Ajk. Starting with the condition
aa-1 = 13
OlrA1A,,
_I
IqJLA 21 A2 2J
2 1AlllA12)
LO
A)
Iq J
34
col
1-- col 3
col 2 - col 1
col 3 -- col 2
42
CHAP. 1
PROBLEMS
(a)
1-42.
c = t
c =al
InTan = al
(b)
(b)
-2
-3
2 3
k = 2, 3_ .. , n
(I
Show that the second, third,.. ., nth columns of c are multiples of the
first column and therefore r(c) < I. When will r(c) = O?
Suppose r(a) = and a
0. In this case, we can write
j-
2, 3 . ., m
Show that the second, third ... , th rows ofec are multiples of the first
row and therefore r(c) < 1. When will r(c) = 0?
1-44. Consider the product
1
al,
C=
... b,]
[ajilaj2] = j[ala2]
3 -1
4
- 2
-1
b
. .lbil
Lb21 b2 2 .. b2,
-10
a l1
/21 a,2
bI=2kl=
7
4
1
2
[bb
a a,, am2
(a)
(a)
3
2
3 -4
43
a2
a2 1
a22
h2
lb,
'1
h22
b,,
...
b2,
(a)
a,n,I anal~~
a2,, ~
Let
1 3
2
7 12 14_
Using (b), we can write (a) as
~ale
[mh5
b.s2
bsnX
c=
2 [B 1B 2
... B,,]
(b)
(c)
mA,
a=
a-=
41 22
2l
7 7 9
Suppose r(a) = r, r(b) = rb. For convenience, we assume the first r rows
of a and the first rb columns of b are linearly independent. Then,
Aj =
~I jAp
J =Zr + 1, a + 2,..., m
44
CHAP. 1
rb
Bk = E
kqBq
PROBLEMS
(b)
k = b + 1, rb + 2,..., n
If m < n and r
m, the equations are consistent for an arbibrary c.
Is this also true when ml > n and r = n? Illustrate for
q=l
(a)
(b)
(c)
{A
45
r-1
IC
i%2
and
r(c)?
(d) To determine the actual rank of c, we must find the rank of
A1B 1
A2B 1
A -,BB
Bb
A1B
A1rb
A2 B2
...
A2Brb
ArB.
, = Ar.B2
ArJ
1-46.
to B 1, B2 ... , B ?
Utilize these resuts to find the rank of
-1/2 1/2 0
- 1/2 1/2
-I
1
(f)
l 0 l
(a)
1 0 l
1 2j
(b)
s. Verify for
L3
i 1 24
2 -22
1-45.
12
21
a22
al. 1
aml 2
X1
C1
j{a2X2
C2
Ct
Cn
(a)
Let
Aj = [ajlaj2 .
ajn]
j = 1, 2 ....,in
(b)
(a)
Ck =
kpCP
k = r + 1, r + 2 .. ,m
p=I
2}
iC3,
xI + x 2 + 2x 3 + 2x 4 = 4
2x + x 2 + 3X 3 + 2 4 =
3X1 + 4x 2 + 2X 3 + X4 = 9
7x + 7 2 + 9 3
7 4
23
Ara'Brb
Suppose r, < rb. What can you conclude about r(c) if Af is orthogonal
(e)