Math 219 Homework 2 Solutions

1. Let
A =
_
3 0
1 1
_
, and B =
_
2
4
_
.
(a) Compute the Kronecker product AB.
(b) Compute the direct sum AB.
Solution:
(a) AB =
_
_
_
_
6 0
12 0
2 2
4 4
_
_
_
_
(b) AB =
_
_
_
_
3 0 0
1 1 0
0 0 2
0 0 4
_
_
_
_
2. Below are several linear models written in scalar notation. Each of these models
can be written in the usual matrix form, namely y = X + . For each of these
models, explicitly write out the design matrix (i.e., give the entries of X); give the
rank of X; and explicitly write out the vector of regression coecients (i.e., give
the entries of in terms of the scalar parameters).
(a) y
i
= +
i
, for i = 1, . . . , 4.
(b) y
i
=
0
+
1
x
i
+
2
x
2
i
+
i
, for i = 1, . . . , 4, where x
1
= 3, x
2
= 2, x
3
= 7, and
x
4
= 4.
(c) y
ij
= + i + i
2
+
ij
, where i = 1, 2, 3; j = 1, . . . , n
i
; n
1
= 1; n
2
= 2; and
n
3
= 3.
Solution:
(a) X =
_

_
1
1
1
1
_

_
, rank(X) = 1, = .
(b) X =
_

_
1 3 9
1 2 4
1 7 49
1 4 16
_

_
, rank(X) = 3, =
_
_

2
_
_
.
1
(c) X =
_

_
1 1 1
1 2 4
1 2 4
1 3 9
1 3 9
1 3 9
_

_
, rank(X) = 3, =
_
_

_
_
.
3. Consider the one-way classication model with k treatments
y
ij
=
0
+
j
+
ij
, i = 1, . . . , n
j
; j = 1, . . . , k.
(a) Give an expression for the design matrix using direct sums.
(b) Suppose that the data are balanced (i.e., n
j
= n for all j). Give an expression
for the design matrix using Kronecker products.
Solution: Let 1
m
be an m1 vector of ones, and N =

k
j=1
n
j
. Then
(a) X = [1
N
1
n
1
1
n
2
1
n
k
]
(b) X = [1
N
I
k
1
n
] or X = [ [1
k
I
k
] 1
n
].
4. For this problem, you are going to examine data on average SAT scores by state
from the years 1994-1995. In particular, we are going to treat sat, the average
total score on the SAT, as our response variable, and frac, the percentage of all
eligible students taking the SAT, as our explanatory variable. Load the data into
R using the following commands:
dat = read.csv("http://www.reed.edu/jones/141/sat.csv")
attach(dat)
The attach command will allow you to refer to the names of variables directly,
e.g., frac, rather than needing the name of the data frame, e.g., dat$frac.
(a) Create a scatter plot of sat vs. frac. Include axes labels and a title. Describe
the relationship between sat and frac. (In addition to Section 4.10 in Hay-
Jahans, you may want to review Lab 2 from last semester:
http://www.math.clarku.edu/shancock/F11/m217/labs/lab2/Graphics.html.)
(b) We are going to t the linear model y
i
=
0
+
1
x
i
+
2
x
2
i
, i = 1, . . . , 50,
where y
i
is the average SAT score for the ith state, and x
i
is the percentage
of all eligible students taking the SAT for the ith state.
i. Create the response vector y in R.
ii. Create the design matrix X in R.
iii. Compute the ordinary least squares parameter estimates,

= (X

X)
1
X

y.
(The R command to compute the inverse of a matrix is solve.)
iv. Verify that your parameter estimates are a solution to the normal equa-
tions by comparing the quantities X

and X

y. (These two quantities

should be equal.)
2
v. Compute the sum of squared error, SSE = (y X

(y X

).
vi. Use the R function lm to t the same linear model. You will need to
create a new vector for the squared term, e.g. x2 = frac^2, then use
this vector in the lm formula. (R doesnt like to do arithmetic within the
lm formula.) Do these parameter estimates match the ones computed in
part (iii.)? Does the SSE match?
(c) Use the R function curve with the option add = T to add the tted quadratic
line to your scatterplot from part (a). Type help(curve) if you are unfamiliar
with this function; there are examples at the bottom of the help page. (If you
closed the plot window, you will have run the plot command again before
the curve command.)
Solution: See Microsoft Word document.
5. Determine whether each of the following sets is a vector space, i.e., check if each
set is closed under vector addition and scalar multiplication. Justify your answer.
(a) V =
_

_
x R
4
; x =
_
_
_
_
a
b
a + b
1
_
_
_
_
, a, b R
_

_
.
(b) V =
_
_
_
x R
3
; x =
_
_
a
b
a 2b
_
_
, a, b R
_
_
_
.
Solution:
(a) Let u, v V and R. Then u =
_
a b a + b 1

and v =
_
c d c + d 1

for some a, b, c, d R. V is not a vector space since

u +v =
_
a + c b + d a + b + c + d 2

/ V
or
u =
_
a b a + b

/ V for = 1.
(b) Let u, v V and R. Then u =
_
a b a 2b

and v =
_
c d c 2d

for some a, b, c, d R. V is a vector space since

u+v =
_
a + c b + d a 2b + c 2d

=
_
a + c b + d (a + c) 2(b + d)

V
and
u =
_
a b (a 2b)

=
_
a b (a) 2(b)

V.
3
6. Consider the vector space
V =
_

_
x; x =
_
_
_
_
a
a + b
a + b
b
_
_
_
_
, a, b R
_

_
.
Determine which of the following sets of vectors are spanning sets and which are
basis sets. Justify your answer.
(a) v
1
=
_
_
_
_
1
0
0
1
_
_
_
_
and v
2
=
_
_
_
_
1
2
2
1
_
_
_
_
(b) v
1
=
_
_
_
_
1
1
0
0
_
_
_
_
and v
2
=
_
_
_
_
0
0
1
1
_
_
_
_
(c) v
1
=
_
_
_
_
2
1
1
1
_
_
_
_
, v
2
=
_
_
_
_
3
1
1
2
_
_
_
_
, and v
3
=
_
_
_
_
3
2
2
2
_
_
_
_
(d) v
1
=
_
_
_
_
1
0
0
0
_
_
_
_
, v
2
=
_
_
_
_
0
1
1
0
_
_
_
_
, and v
3
=
_
_
_
_
0
0
0
1
_
_
_
_
Solution:
(a) First check that the vectors are in V : v
1
V by setting a = 1 and b =
1; v
2
V by setting a = 1 and b = 1. Now let x V . Then x =
_
a a + b a + b b

for some a, b R. Try to solve x = c

1
v
1
+ c
2
v
2
for c
1
and c
2
:
_

_
a
a + b
a + b
b
_

_
= c
1
_

_
1
0
0
1
_

_
+ c
2
_

_
1
2
2
1
_

_
=
c
1
+ c
2
= a (1)
2c
2
= a + b (2)
2c
2
= a + b (3)
c
1
c
2
= b (4)
4
The equations (2) and (3) give us c
2
= (a +b)/2. Substituting this into equa-
tion (1) gives c
1
= (a b)/2. Plugging c
1
= (a b)/2 and c
2
= (a + b)/2 into
the original equation veries the solution. Thus, {v
1
, v
2
} is a spanning set for
V .
Since v
2
= kv
1
for any k R, v
1
and v
2
are linearly independent, and {v
1
, v
2
}
is a basis set for V .
(b) v
1
/ V and v
2
/ V (their second and third elements do not match). Therefore,
{v
1
, v
2
} cannot be a spanning set (or basis set) for V .
(c) v
1
V (a = 2 and b = 1), v
2
V (a = 3 and b = 2), but v
3
/ V .
Therefore, {v
1
, v
2
, v
3
} cannot be a spanning set (or basis set) for V .
(d) v
1
, v
2
, v
3
/ V (check that each rst element minus the fourth element is not
equal to the second and third elements). Therefore, {v
1
, v
2
, v
3
} cannot be a
spanning set (or basis set) for V .
7. Let X be an n p matrix with rank r.
(a) Dene the vector space C(X).
(b) Dene the vector space N(X

).
(c) Dene the vector space {C(X

)}

.
Solution:
(a) C(X) = {u R
n
: u = Xb for some b R
p
}.
(b) N(X

) = {v R
n
: X

v = 0}.
(c)
{C(X

)}

= {w R
p
: w

u = 0 for any u C(X

)}
= {w R
p
: w

b = 0 for any b R
n
}
= {w R
p
: b

Xw = 0 for any b R
n
}
= {w R
p
: Xw = 0}
= N(X).
8. Consider the linear model
y = X + ,
where X is n p with rank r. Let

be any solution to the normal equations
X

X = X

y. Dene y and e as y = X

and e = y y.
(a) What vector spaces to y, y, and e live in?
(b) Verify that y e.
5
Solution:
(a) y R
n
, y = X

C(X), and e N(X

). To show e N(X

), dene
P = X(X

X)

. Then
X

e = X

(y y
= X

(y Py)
= (X

P)y
= 0
since PX = X = X

= X

= X

p = X

(see Theorems 11.21 and

11.23 on p. 90-91 of Chapter 11 in Dr. Boiks lecture notes).
(b) By the fundamental theorem of linear algebra, C(X) = [N(X

)]

; thus, y e.
Also, since P

= P and P
2
= P,
y

e = y

(I P)y = y

P(I P)y = y

(PP
2
)y = y

(PP)y = 0.
6