Problem Set 1: Unbiased Estimator of the Error

Variance
This assignment will guide you through the derivations needed to determine what is
a unbiased estimator of the error variance in the context of a univariate linear regression.
This assignment may be quite challenging. Good Luck!
Consider the univariate linear regression model
yt = + xt + ut ,

t = 1, . . . , T.

(1)

where the regressors are non-stochastic (fixed) and the disturbances have zero mean
and are uncorrelated and homoscedastic with variance equal to 2 , i.e., E [ut ] = 0,
C [ut , us ] = 0, t 6= s, and V [ut ] = E [u2t ] = 2 . The aim of this problem set will be to
derive that
T
1 X 2
2
u
s =
T 2 t=1 t
t,
is an unbiased estimator of the disturbance variance 2 , where ut = yt yt , yt =
+ x
and
and are the OLS estimators of and , respectively. The OLS estimates

and are given by

=

PT

(yt y) (xt x)
,
PT
)2
t=1 (xt x

= y x,

t=1

which can also be expressed as

=

T
X

wt yt ,

t=1

where
wt = PT

qt y t ,

t=1

xt x

t=1

T
X

(xt x)

qt =

1
x wt .
T

Question 1 - Verify simple properties.

Verify the following 7 properties:
PT
1.
t=1 wt = 0
2.

PT

w t xt = 1

3.

PT

=1

4.

PT

5.

PT

6.

PT

7.

PT

t=1

t=1 qt

t=1 qt xt

t=1

=0
1

wt2 =

2
t=1 qt =
t=1 qt wt

PT

x)2
t=1 (xt

PT

t=1

PTT

x2t

x)2
t=1 (xt

x
PT

x)2
t=1 (xt

As you work through the problem set, you should always look back at these 8 properties
and see which one can help you in each step.
For example, with properties 1 and 2 in hand, it is easy to show (as we did in class)
that:
T
T
T
X
X
X

=
wt yt =
wt ( + xt + ut ) = +
wt ut .
t=1

t=1

t=1

Likewise, using properties 3 and 4 it is easy to show (as we did in class) that:

T
X

qt yt =

t=1

T
X

qt ( + xt + ut ) = +

t=1

T
X

qt ut .

t=1

Question 2 - Computing the expression for the sum of

the residuals.
Show that the residuals from the regression described by Eq. 1 can be expressed as:
h

 i
ut = (
) + xt + ut .
And, therefore,
u2t

h

 i
 i2
2

= (
) + xt + ut 2 (
) + xt ut
h

The sum of the residuals therefore can be expressed as:

T
X

u2t

T h
X

t=1

T
T
 i2 X
h

 i
X
2

(
) + xt +
ut
2 (
) + xt ut

t=1

t=1

t=1

The expected sum of the residuals therefore can be expressed as:

T
X

h
T
T
T
h h

 i i

 i2  X
  X
 2 X
E 2 (
) + xt ut
E (
) + xt
+
E u2t
E ut =
t=1

t=1

t=1

t=1

The expression above can be broken down into 3 components:

h h

 i i
P
) + xt ut
Component 1: Tt=1 E 2 (
h

 i2 

(
) + xt
t=1 E

Component 2:

PT

Component 3:

PT

t=1

E [u2t ]

The next 3 questions will make you work on each of these components.

Question 3 - Working with Component 1.

Note that:


(
) + xt =

T
X
s=1

!
qs u s

T
X

!
ws us

s=1

Using the expression above, show that:

hh
E


 i i
(
) + xt ut = (qt + wt xt ) 2 ,

and, therefore, that

T
X

h h

 i i
E 2 (
) + xt ut = 4 2 .

t=1

xt .

Question 4 - Working with Component 2

Show that
"
#
PT
h
1

 i2 
2
2
x
x
x

x
t
t
E (
) + xt
= 2 PTT t=1 t 2 + PT
2 PT
,
2
2
(x

)
(x

)
(x

)
t
t
t
t=1
t=1
t=1
and, therefore, that
T
X

h
E

 i2 

(
) + xt
= 2 2 .


t=1

(This can be quite challenging)

Question 5 - Working with Component 3

Show that
T
X

 
E u2t = T 2

t=1

(This should be very easy)

Final Step
This final step is easy and I will solve it for you. Once we have obtained the expressions
for the three components in questions 3, 4, and 5 we simply have to put all the pieces
together to obtain an unbiased estimator of the error variance. To be precise:
T
X
t=1

u2t

T
T
T
h

 i2 X
h h

 i i
X
 2 X

=
E (
) + xt +
E ut
E 2 (
) + xt ut
t=1

t=1

t=1

simplifies to
T
X

 
E u2t = 2 2 + T 2 4 2 = (T 2) 2 .

t=1

Divide by (T 2) on both sides the expression above to obtain the unbiased estimator
of the error variance:
#
"
T
1 X 2
u = 2 .
E
T 2 t=1 t