Sie sind auf Seite 1von 17

Matrix Algebra Review

A matrix is a mathematical way or organizing information that can be arranged as rows and col-
umns. For many types of tables of data, these matrices can simplify the mathematical formulae
used to express various multivariate analyses. In fact, most multivariate texts will present formu-
las for the data analysis and theory in terms of matrix algebra. For this reason, we spend time in
this unit reviewing and introducing some basic concepts and basic operations of matrix algebra.
These concepts and operations will be introduced with reference to a multivariate data example.

The data set consists of 4 measurements made on the same skulls of 2 groups of individuals,
males and females. Thus, an observation consists of the 4 skull measurements on a given skull
and of the sex of the individual to which the skull belonged. We might have several objectives in
the data analysis including a description (means, medians, variances, covariances, correlations) of
the measurements according to the sex of the individual and an hypothesis test to determine if the
skulls differ in size or shape between the sexes. The data are presented in Table 1 while various
data summaries are given in Tables 2-6.
TABLE 1. Four skull measurements on males and females.

Zygomatic
Sex Length Basilar Arch Postorbital
F 6287 4845 3218 996
F 6583 4992 3300 1107
F 6518 5023 3246 1035
F 6432 4790 3249 1117
F 6450 4888 3259 1060
F 6379 4844 3266 1115
F 6424 4855 3322 1065
F 6615 5088 3280 1179
F 6760 5206 3337 1219
F 6521 5011 3208 989
F 6416 4889 3200 1001
F 6511 4910 3230 1100
F 6540 4997 3320 1078
F 6780 5259 3358 1174
F 6336 4781 3165 1126
F 6472 4954 3125 1178
F 6476 4896 3148 1066
F 6276 4709 3150 1134
F 6693 5177 3236 1131
F 6328 4792 3214 1018
F 6661 5104 3395 1141
F 6266 4721 3257 1031

Matrix Algebra Review 2


TABLE 1. Four skull measurements on males and females.

Zygomatic
Sex Length Basilar Arch Postorbital
F 6660 5146 3374 1069
F 6624 5032 3384 1154
F 6331 4819 3278 1008
F 6298 4683 3270 1150
M 6460 4962 3286 1100
M 6252 4773 3239 1061
M 5772 4480 3200 1097
M 6264 4806 3179 1054
M 6622 5113 3365 1071
M 6656 5100 3326 1012
M 6441 4918 3153 1061
M 6281 4821 3133 1071
M 6606 5060 3227 1064
M 6573 4977 3392 1110
M 6563 5025 3234 1090
M 6552 5086 3292 1010
M 6535 4939 3261 1065
M 6573 4962 3320 1091
M 6537 4990 3309 1059
M 6302 4761 3204 1135
M 6449 4921 3256 1068
M 6481 4887 3233 1124
M 6368 4824 3258 1130
M 6372 4844 3306 1137
M 6592 5007 3284 1148
M 6229 4746 3257 1153
M 6391 4834 3244 1169
M 6560 4981 3341 1038
M 6787 5181 3334 1104
M 6384 4834 3195 1064
M 6282 4757 3180 1179
M 6340 4791 3300 1110
M 6394 4879 3272 1241
M 6153 4557 3214 1039
M 6348 4886 3160 991
M 6534 4990 3310 1028
M 6509 4951 3282 1104

Matrix Algebra Review 3


TABLE 2. Summary statistics output from SAS PROC CORR for the skulls of females.

Variable N Mean Std Dev Sum Minimum Maximum


LENGTH 26 6486 152.2867 168637 6266 6780
BASILAR 26 4939 158.9637 128411 4683 5259
ZYGOMAT 26 3261 73.7245 84789 3125 3395
POSTORB 26 1094 64.4921 28441 989 1219

TABLE 3. Summary statistics output from SAS PROC CORR for the skulls of males.

Variable N Mean Std Dev Sum Minimum Maximum


LENGTH 33 6429 185.6793 212162 5772 6787
BASILAR 33 4898 149.1453 161643 4480 5181
ZYGOMAT 33 3259 62.9397 107546 3133 3392
POSTORB 33 1090 53.3801 35978 991.0000 1241

TABLE 4. Variances and covariances output from SAS PROC CORR for the skulls of females.

LENGTH BASILAR ZYGOMAT POSTORB

LENGTH 23191.23846 23352.12462 6580.63538 4946.08462

BASILAR 23352.12462 25269.46615 6272.09385 3895.38615

ZYGOMAT 6580.63538 6272.09385 5435.30615 1189.57385

POSTORB 4946.08462 3895.38615 1189.57385 4159.22615

TABLE 5. Variances and covariances output from SAS PROC CORR for the skulls of males.

LENGTH BASILAR ZYGOMAT POSTORB

LENGTH 34476.82008 26469.89489 6924.72348 -1539.66288

BASILAR 26469.89489 22244.32955 5232.72727 -1852.44318

ZYGOMAT 6924.72348 5232.72727 3961.40530 104.10133

POSTORB -1539.66288 -1852.44318 104.10133 2849.43939

TABLE 6. Pearson correlations output from SAS PROC CORR. Correlations for the skulls of females
are given in the upper right triangle while correlations for the skulls of males are given in the lower left
triangle.

Males \ Females LENGTH BASILAR ZYGOMAT POSTORB

LENGTH 1 0.96464 0.58613 0.50361

BASILAR 0.95583 1 0.53518 0.37997

ZYGOMAT 0.59254 0.55743 1 0.25019

POSTORB -0.15534 -0.23268 0.03099 1

Matrix Algebra Review 4


1. A vector is a directed line in p -dimensions. We can think of the components of the vector as
being separate axes for plotting the vector and the values of the components as being how
far along the axis from the origin that we travel in locating the end of the vector. For exam-

ple, let x = 1 be a vector whose x 1 element is 1 and whose x 2 element is 2. We can plot
2
this vector in 2-space ( p = 2 ) by simply locating the end-point or head of the vector
according to the axis specifications above and then drawing a line from the origin out to this
end-point.
x1

0
x2
0 1 2
Vectors can be specified as row vectors, such as x = 1 2 , or as column vectors, as defined
above. This distinction becomes important later on when we perform operations on the vec-
tors. Here, we have written the vector with an underscore to emphasize that it is a vector and
not a scalar. For the most part, variables x , y , and z will refer to vectors.
2. A column vector can be changed to a row vector and vice versa using the transpose operator.

Let x = 1 then x = 1 2 .
2
3. Multiplication of a vector by a positive constant expands or contracts the vector. Multiplica-
tion of a vector by a negative constant also expands or contracts the vector but also changes
the direction of the vector by 180 degrees (opposite direction).

3 1 = 3 or ( 3 ) 1 = 3
2 6 2 6
4. Addition and subtraction of vectors is also possible. The vectors must be conformable, i.e.,
they must both either be row vectors or column vectors and they must have the same num-
ber of elements. Vector addition and subtraction are performed by adding or subtracting the
corresponding elements from each vector.

x1 y1 x1 + y1
Let x = and y = then x + y = .
x2 y2 x2 + y2

Matrix Algebra Review 5


1 + 3 = 4 and
12 + 34 = 46 .
2 4 6

Lets estimate the population average skull size by ignoring the sex to
which a skull belongs. Thus, we need a weighted average of the vectors of
means of the 2 sexes.
6486 6429 6454.1
-----------------
26 4939 33 4898 = 4916.1
26 + 33- + ------------------
26 + 33
3261 3259 3259.9
1094 1090 1091.8
5. The length of a vector can be computed using the Pythagorean theorem which can be gener-
alized to p -dimensions. Let x = x 1 x 2 x p then the length of x is

x = x 12 + x 22 + + x p2 .

Let x be the vector of postorbital measurements for the females. Then the
length of this vector would be
x = 996 2 + 1107 2 + + 1150 2 = 5587.05 .
6. The usual inner product of two conformable vectors x and y is defined to be
xy = x 1 y 1 + x 2 y 2 + + x p y p . Thus the length of a vector can be written in terms of its
inner product with itself as x = xx .

6287 6583
Let x = 4845 and y = 4992 then
3218 3300
996 1107

6583
xy = 6287 4845 3218 996 4992
3300
1107

= 6287 ( 6583 ) + 4845 ( 4992 ) + 3218 ( 3300 ) + 996 ( 1107 ) = 77295533


7. The cosine of the geometric angle between two conformable vectors x and y is given by
xy xy
cos = --------------- = ----------------------- .
x y xx yy

Using the vectors from the previous example, xy = 77295533 ,


xx = 74347934 , and yy = 80371402 , so that

Matrix Algebra Review 6


77295533
cos = -------------------------------------------------------- = 0.99992902 and
74347934 80371402
= acos 0.99992902 = 0.01191494 . Thus, the vectors are nearly col-
linear.
Now lets let the vector x be the vector of basilar measurements for the
skulls of the females and well let the vector y be the corresponding pos-
torbital measurements on the same skulls. For each measurement, subtract
away the mean value of the measurements for that variable. Thus, from
each of the values of x we will subtract 4938.885 and from each of the
values of y we will subtract 1093.885 . This will produce 2 centered vec-
tors. This process will be discussed later when we study the sample geome-
try. Now compute the 3 inner products using these 2 centered vectors, x
and y .
xy = 97384.654 , xx = 631736.65 , and yy = 103980.65 so that
97384.654
cos = ----------------------------------------------------------- = 0.3799673 and
631736.65 103980.65
= acos 0.3799673 = 1.1810354 radians or 67.668 degrees.
Compare the cosine of the angle between these centered basilar and pos-
torbital measurements with the correlation coefficient between these same
2 variables for the skulls of the females.
8. Two vectors x and y are perpendicular (orthogonal) if xy = 0 . This follows from the
statements above about the angle between two vectors. Simply specify that the angle is to be
90 degrees and require that the lengths of both vectors be greater than 0, then the only way
that the equation can be satisfied is for the inner product of x with y to be zero. What does
this imply about the correlation between two variables whose centered vectors are perpen-
dicular? If a set of 3 or more vectors are all pair-wise perpendicular, then they are said to be
mutually orthogonal or mutually perpendicular.
9. A set of vectors x 1, x 2, , x k is linearly dependent if there exists constants c 1, c 2, , c k ,
not all zero, such that c 1 x 1 + c 2 x 2 + + c k x k = 0 . Linear dependence implies that one of
the vectors can be written as a linear combination of the others. This would be the case if,
say, one variable is the sum of 2 other variables.

1 3 5
Are the vectors x = 1 , y = 4 , and z = 6 linearly dependent?
1 1 3

Write the system of equations using the definition.

Matrix Algebra Review 7


1 3 5 0 c 1 + 3c 2 + 5c 3 = 0
c 1 1 + c 2 4 + c 3 6 = 0 or c 1 + 4c 2 + 6c 3 = 0 .
1 1 3 0 c 1 + c 2 + 3c 3 = 0

Solve the system of equations by, for example, subtracting the first equa-
tion from the second, and then the third equation from the second,

c2 + c3 = 0
c 2 + c 3 = 0 or that c 2 = c 3 .
3c 2 + 3c 3 = 0

Since we are trying to show linear dependence, pick some non-zero val-
ues for c 2 and c 3 that fit the last equation. For example, take c 2 = 1 and
c 3 = 1 . Using any one of the original equations, this implies that
c 1 = 2 . Thus, these 3 constants are non-zero and they solve all 3 equa-
tions. Therefore, the 3 vectors must be linearly dependent. One can be writ-
ten as a linear combination of the other 2. What this means in terms of
geometry is that although there are 3 vectors and each has 3 coordinate val-
ues (3 axis values), these 3 vectors can actually be plotted in fewer than 3
dimensions by finding the appropriate coordinate system (Here, 2 axes. In
linear algebra this minimal set of axes is called a basis).

1 3 3
Are the vectors x = 1 , y = 4 , and z = 6 linearly dependent?
1 1 5

Write the system of equations using the definition.

1 3 3 0 c 1 + 3c 2 + 3c 3 = 0
c 1 1 + c 2 4 + c 3 6 = 0 or c 1 + 4c 2 + 6c 3 = 0 .
1 1 5 0 c 1 + c 2 + 5c 3 = 0

Solve the system of equations by, for example, subtracting the first equa-
tion from the second, and then the third equation from the second,

c 2 + 3c 3 = 0
c 3 = 0 and that c 2 = 0 and that c 1 = 0 .
3c 2 + c 3 = 0

Since the only values for the c i that will solve the equations are c i = 0 ,
the vectors are linearly independent. Note: this does not imply that they are
mutually perpendicular.

Matrix Algebra Review 8


xy
10. The projection or shadow of the vector y on the vector x is given by Proj(y x) = ------- x .
xx
Notice that it is a vector along x but whose length has been changed to correspond to the
shadow of y produced on x .

Proj(y x)
x

The lengths of these projections will often correspond to regression coefficients in normal
theory linear and multiple regression ( = ( XX ) 1 ( Xy ) where the vector y is projected
into the space of the predictor variables).
11. A matrix X is a transformation of the rows ( n ) to columns ( p ) and columns to rows. A
( nxp )
very common representation is the data matrix whose rows are the observations and whose
columns are the variables. This would be the data of Table 1. Typically, we would have a
separate matrix for males and females and would discard the column corresponding to sex.
The other tables could also be described as matrices.
12. A square matrix has the same number of rows as columns, n = p . The covariance matrix
and correlation matrix (Tables 4, 5, and 6) are square matrices.
13. The transpose of a matrix X is the new matrix X whose rows are the columns of X and
whose columns are the rows of X , such that x ij = x ji , where i and j are the matrix indices
for rows and columns with the row index listed first.

If = 1 3 then = 1 4 .
4 2 3 2
14. The matrix X is a square symmetric matrix if X = X . Notice that the covariance matrix and
correlation matrix are symmetric matrices. Note also that Table 6 is actually 2 different cor-
relation matrices written as one by exploiting this fact.
15. The identity matrix I is a square symmetric matrix that has 1s on the main diagonal
( i jj = 1 for j = 1, 2, , n ) and 0s elsewhere.

1 0 0
For n = 3 , = 0 1 0 .
0 0 1

The identity matrix has the property XI = IX = X where I is conformable with X.


16. A diagonal matrix is a square matrix whose off-main-diagonal elements are all zero. The
identity matrix is a special diagonal matrix whose main diagonal elements are all 1.

Matrix Algebra Review 9


17. The rank of a matrix is the number of linearly independent columns of the matrix. Note that
the rank can be no larger than the larger of the number of rows or the number of columns of
the matrix. The row rank and the column rank of a matrix are the same.
18. If the matrix X is multiplied by a scalar constant, then each element of X is multiplied by
the constant.

If = 1 3 then 2X = 2 6 .
4 2 8 4
19. Two matrices can be added if they are conformable for addition, i.e., they have the same
number of rows and columns. Addition is performed by adding together the corresponding
elements of each matrix and placing the results in the corresponding elements of a new
matrix of the same dimensions.
Compute the pooled variance-covariance matrix of the skull measurements
by pooling the variance-covariance matrices of the males and females.
Note that we must take into account the degrees of freedom of each matrix.
( nm 1) Sm + ( nf 1) Sf
S pool = -----------------------------------------------------------
- where S m and S f are the variance-
nm + nf 2
covariance matrices for the males and females, respectively, and, n m and
n f are the number of skulls for the males and females, respectively. To save
some space with this example, the matrix entries will be rounded to integer
numbers.

34477 26470 6925 1540 23191 23352 6581 4946


( 33 1 ) 26470 22244 5233 1852 + ( 26 1 ) 23352 25269 6272 3895
6925 5233 3961 104 6581 6272 5435 1190
1540 1852 104 2849 4946 3895 1190 4159
S pool = ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
33 + 26 2

1103264 847040 221600 49280 579775 583800 164525 123650


847040 711808 167456 59264 + 583800 631725 156800 97375
221600 167456 126752 3328 164525 156800 135875 29750
49280 59264 3328 91168 123650 97375 29750 103975
= ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
57

1683039 1430840 386125 74370


1430840 1343533 324256 38111
386125 324256 262627 33078
74370 38111 33078 195143
= ------------------------------------------------------------------------------------------
57

Matrix Algebra Review 10


29527 25102.456 6774.1228 1304.7368
= 25102.456 23570.754 5688.7018 668.61404
6774.1228 5688.7018 4607.4912 580.31579
1304.7368 668.61404 580.31579 3423.5614

20. Matrix multiplication can also be performed if the matrices are conformable. Let X be a
p n matrix and let Y be a n k matrix, then the product XY is defined and the resulting
matrix will have dimensions p k . The matrix multiplication is essentially a set of inner
products of the rows of the first matrix with the columns of the second matrix with the result
being entered into the new matrix at the position defined as the row of the first matrix and
the column of the second matrix that are in the inner product. Note that the order of the
matrices in the operation is very important. Note also that pre- or post-multiplication of a
matrix by a conformable identify matrix returns the original matrix.

If = 1 3 and Y = 5 2 then
4 2 1 3

XY = 1 3 5 2 = 1 ( 5 ) + 3 ( 1 ) 1 ( 2 ) + 3 ( 3 ) = 8 7
4 2 1 3 5 ( 4) + 2 ( 1) 4 ( 2) + 2 ( 3) 18 14
21. The trace of a matrix is the sum of its diagonal elements.

If = 1 3 then tr(X) = tr( 1 3 ) = 1 + 2 = 1 .


4 2 4 2

The trace of a sample variance-covariance matrix is often called the total


sample variance as it is the sum of the variances of each of the variables.
Find the total sample variance of S pool .

29527 25102.456 6774.1228 1304.7368


tr(S pool) = tr( 25102.456 23570.754 5688.7018 668.61404 )
6774.1228 5688.7018 4607.4912 580.31579
1304.7368 668.61404 580.31579 3423.5614
= 29527 + 23570.754 + 4607.4912 + 3423.5614 = 61128.807
22. The determinant of the square matrix X, X , can be defined by the recursive formula
k
X = x1j X1j ( 1 ) 1 + j for k > 1 and X = x 11 for k = 1 , where X 1j is the matrix
j=1
formed by deleting the 1st row and jth column of the specified matrix, and k is the number
of rows and columns of the matrix. For a 2 2 matrix, this formula reduces to
X = x 11 x 22 x 12 x 21 .

Matrix Algebra Review 11


If = 1 3 then X = 1 3 = 1 ( 2 ) 3 ( 4 ) = 14 .
4 2 4 2

The determinant of a sample variance-covariance matrix is also used as a


measure of multivariate variability and is called the generalized sample
variance. Find the generalized sample variance of S pool

29527 25102.456 6774.1228 1304.7368


det(S pool) = det( 25102.456 23570.754 5688.7018 668.61404 ) = 6.5427 10
14
6774.1228 5688.7018 4607.4912 580.31579
1304.7368 668.61404 580.31579 3423.5614
as computed using a computer subroutine in SAS/IML.

23. The matrix inverse of the square matrix X is the matrix X 1 such that X 1 X = XX 1 = I .
For the inverse to exist, the columns of X must be linearly independent (full rank). If the
matrix is not of full rank then the matrix is called singular, while if it is of full rank then it
is called non-singular.

1 x x 12
For a 2 2 matrix X , the inverse is X 1 = ------ 22 where X is the
X x x
21 11
determinant of the matrix.

If = 1 3 then
3 2
1
1
X 1 = 1 3 = --------- 2 3 = 0.18181818 0.27272727 .
3 2 11 3 1 0.27272727 0.09090909

Now, lets verify this result,

X 1 X = 0.18181818 0.27272727 1 3 = 1 0 and so we have


0.27272727 0.09090909 3 2 0 1
found the inverse of the matrix X .
In statistics, the inverse of the variance-covariance matrix is called the
concentration matrix. When appropriately standardized, it contains the
partial correlations of each pair of variables adjusted for every other vari-
able. The concentration matrix of S pool is

Matrix Algebra Review 12


0.0003905 0.000395 0.000079 0.00058
1
S pool = 0.000395 0.0004604 0.00000507 0.0000599
0.000079 0.00000507 0.00033 0.000027
0.000058 0.0000599 0.000027 0.0003072

Note that the inverse of a square symmetric matrix is also square and sym-
metric. Further note that in many statistical applications due to the high
correlations among variables, the inverse can be very sensitive to rounding
errors. Thus, it is important to use good subroutines and statistical software
to avoid numerical problems.

24. An orthogonal matrix is a matrix Q such that QQ = QQ = I so that Q = Q 1 .


25. A square matrix A , with k rows and columns, has an eigenvalue (characteristic root) with
corresponding eigenvector (characteristic vector) x , x 0 , if Ax = x . The normalized
eigenvectors are often written e i and have length 1. A will have k pairs of eigenvalues and
eigenvectors, ( 1, e 1 ) , ( 2, e 2 ) , , ( k, e k ) . The eigenvectors are unique unless 2 or
more eigenvalues are equal. The distinct eigenvectors of a symmetric matrix are mutually
perpendicular. Further, the sum of the eigenvalues of a matrix equals its trace, and the prod-
uct of the eigenvalues of a matrix equals its determinant.

The above results also imply the characteristic equation, A I = 0 , which can be used to
find the eigenvalues of A .

Let = 1 0 . Find the eigenvalues using the characteristic equation.


13

A I = 0 1 0 1 0 = 0 1 0 0 = 0
13 0 1 1 3 0

1 0 = 0 ( 1 ) ( 3 ) ( 1) ( 0) = 0
1 3
( 1 ) ( 3 ) = 0 . The roots of this quadratic equation can be
found using the quadratic formula or, here, by inspection. Thus, 1 = 1
and 2 = 3 are the eigenvalues.

Find the corresponding eigenvectors using the relationship Ax = x .

1 0 x1 = x1
x 1 = x1

x 1 = x1
. For exam-
1
1 3 x2 x2 x 1 + 3x 2 = x 2 x 1 = 2x 2
x1 = 2
ple, is a solution. The normalized eigenvector e 1 can be found
x2 = 1

Matrix Algebra Review 13


2 5
-------------
by dividing the elements of x by the length of x . Thus, = 5 . Sim-
1
5
-------
5
x 1 = 3x1
ilarly for 2 , we get the system of equations
x 1 + 3x 2 = 3x 2
x1 = 0 x1 = 0
with a solution . The normalized eigenvector e 2 is
x2 = x2 x2 = 1

2 = 0 . Note, we couldnt pick anything but 0 for element x 1 .


1
In practice, the eigenvalues and eigenvectors should be computed using
proven subroutines and software.
Since the eigenvalues are related to the trace and determinant of a matrix,
then they must also be measures of variability when computed on a vari-
ance-covariance matrix. In fact, the eigenvectors are a basis set of vec-
tors for summarizing or plotting the data and the eigenvalues are the
corresponding variabilities of the data along the directions given by the
eigenvectors. This is essentially what a principal components analysis
does. Find the eigenvectors and eigenvalues of S pool .

53474.014
= 3641.7324
2824.7904
1189.3712

Find the eigenvectors of S pool .

0.7359715 0.0357156 0.093568 0.669564


E = 0.652417 0.211027 0.104063 0.72041
0.1783248 0.4818466 0.8517509 0.1026861
0.0299708 0.8497152 0.504912 0.1488272

You should also note that the matrix of eigenvectors of a square, symmetric
matrix is also an orthogonal matrix,

Matrix Algebra Review 14


0.7359715 0.652417 0.1783248 0.0299708 0.7359715 0.0357156 0.093568 0.669564
EE = 0.0357156 0.211027 0.4818466 0.8497152 0.652417 0.211027 0.104063 0.72041
0.093568 0.104063 0.8517509 0.504912 0.1783248 0.4818466 0.8517509 0.1026861
0.669564 0.72041 0.1026861 0.1488272 0.0299708 0.8497152 0.504912 0.1488272

-8 -7 -7
0.99999997 9.566176 10 1.5926028 10 3.4196496 10
-8 -7 -7
= 9.566176 10 1.0000001 1.3958974 10 1.589027 10
-7 -7 -8
1.5926028 10 1.3958974 10 0.9999998 7.740809 10
-7 -7 -8
3.4196496 10 1.589027 10 7.740809 10 1.0000005

which is an identity matrix except for rounding errors.


26. A symmetric k k matrix X can be decomposed into a form consisting only of its eigenval-
ues and eigenvectors. This is called the spectral decomposition of X .

X = PD P = 1 e 1 e 1 + 2 e 2 e 2 + + k e k e k where P is an orthogonal matrix whose

columns are the eigenvectors of X , P = e 1 e 2 e k , and D is a diagonal matrix whose

1 0 0
0 2 0
main diagonal elements are the corresponding eigenvalues of X , D = .
. . . .
0 0 k

e1
e2
You should also note that = .
.
ek
As an example, use the matrix E above for the matrix P , then construct the
diagonal matrix D as

53474.014 0 0 0
D = 0 3641.7324 0 0 .
0 0 2824.7904 0
0 0 0 1189.3712

The product ED E will return S pool .

Matrix Algebra Review 15


27. Using the spectral decomposition of the same matrix from above, it can be shown that
k
1
X 1 = PD 1 P = --- i ei ei where, again, i and ei , are the corresponding eigenvalues
i=1
and eigenvectors of X . Note that this also implies that for the inverse of X to exist, all of the
i must be non-zero. Note also that the eigenvectors of X and of X 1 are the same.

28. The form xAx , where x is a vector and A is a symmetric matrix, is called a quadratic form
as it involves square and cross-product terms when expanded.

If xAx 0 for all x 0 , then A is said to be non-negative definite or positive semi-definite.

If xAx > 0 for all x 0 , then A is said to be positive definite.

If A is positive definite, then all of the eigenvalues of A are positive. If A is non-negative


definite, then all of the eigenvalues of A are non-negative, although some may be zero. One
can show these results by writing A in terms of its spectral decomposition.
Quadratic forms are used to measure statistical distance between two
points, such as two means. Mahalanobis distance would standardize the
squared Euclidean distance between two points by the variance-covariance
matrix of the difference. Compute the squared (Mahalanobis) distance
between the mean vectors of measurements of the skulls of males and
females.

T 2 = ( y m y f ) ------ + ---- S pool


1 1 1
( y m y f ) = 2.579
nm nf

Notice that this is a quadratic form where x = ( y m y f ) and

A = -----
1
- + ---- S
1 1
in our earlier formula. Also note that in order for
n m n f pool

distances to always be positive when y m and y f do not coincide, the vari-


ance-covariance matrix S pool must be positive definite. To evaluate the sta-
tistical significance of the squared distance T 2 = 2.579 , we need to know
the distribution of T 2 . This is covered under Hotellings T-square test.
29. A rectangular matrix X of dimensions n p can be written in terms of its singular value
decomposition, X = UD V , where U is n p with orthogonal columns, V is p p and
is orthogonal, and D is p p and is diagonal. The elements of D , 1, 2, , p , are
called the singular values of X .

Note that the spectral decomposition of XX is XX = V ( D ) 2 V and the spectral decom-

Matrix Algebra Review 16


position of XX is XX = U ( D ) 2 U . Here, we are writing X in terms of its singular value
decomposition and simplifying. For example,
XX = ( UD V ) ( UD V ) = ( VD U ) ( UD V ) making use of the rule that when we
transpose the product of matrices, we reverse the order and transpose each component.
Next, ( VD U ) ( UD V ) = VD UUD V = VD ID V = V ( D ) 2 V by realizing
that D is symmetric so that D = D , and U is orthogonal so that UU = I , and
ID = D since I is the identity matrix.

Further, these results imply that the square of the singular values of X are the non-zero
eigenvalues of XX and XX assuming that X is of full column rank.

Matrix Algebra Review 17


Consider the singular value decomposition of the matrix of measurements
for the skulls of females.

45131.02 0 0 0
D = 0 377.19 0 0
0 0 297.96 0
0 0 0 148.97

0.7329997 0.00708 0.1330358 0.667055


V = 0.5582613 0.549046 0.25057 0.5693047
0.3684694 0.8351434 0.199715 0.3562018
0.1236364 0.0321553 0.9378861 0.3225674

U =
0.1910442 0.039454 -0.289166 0.2150759
0.1986437 0.0109427 0.0138016 -0.112158
0.1973333 -0.15866 -0.231725 0.0123482
0.1933035 0.1957425 0.1819122 -0.308274
0.1947336 0.0700373 -0.078581 -0.113865
0.193244 0.1556038 0.0951482 0.171743
0.1944312 0.2584742 -0.088927 0.0379134
0.2003849 -0.16754 0.1873947 0.2195149
0.2047745 -0.212409 0.2406047 0.2440917
0.1967973 -0.229306 -0.339616 -0.237416
0.1935504 -0.066441 -0.240768 -0.226629
0.1958693 -0.02393 0.0754968 -0.285667
0.198091 0.0462814 -0.114289 0.0845266
0.2058031 -0.247272 0.049245 0.3098562
0.1909718 0.0254284 0.2312493 -0.094161
0.1951365 -0.313074 0.3369767 -0.025048
0.194365 -0.187348 0.0195847 -0.452141
0.1890062 0.0988286 0.3002431 -0.119194
0.2022619 -0.400064 0.0257814 0.0012229
0.1910822 0.1088505 -0.154361 -0.132992
0.2021647 0.0596883 -0.002213 0.2673777
0.1895836 0.3096762 -0.110239 0.004265
0.2022993 -0.054062 -0.250534 0.2262461
0.2006189 0.1419394 0.090107 0.1597449
0.19196 0.2103423 -0.250101 0.0881389
0.1900654 0.4033163 0.3018608 0.0045121

Matrix Algebra Review 18

Das könnte Ihnen auch gefallen