OLS Under Heteroskedasticity

Testing for Heteroskedasticity

Heteroskedasticity and Weighted Least Squares

Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009

April 14, 2009

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

The Classical Linear Model:

1 Linearity: Y = X + u.
2 Strict exogeneity: E(u) = 0
3 No Multicollinearity: (X) = K.
4 No heteroskedasticity/ serial correlation: V (u) = 2 In .

Gauss/Markov Theorem: = (X 0 X)1 X 0 Y is best linear

unbiased.
= S 2 (X 0 X)1 is an unbiased estimate of
V ()
= 2 (X 0 X)1 .
V ()

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

What happens if we drop the homoskedasticity assumption?

(the OLS estimator) is still linear and unbiased (Why?).

Though linear and unbiased, is not the minimum variance
estimate (inefficient).
= S 2 (X 0 X)1 is biased. This makes standard t and
V ()
F tests invalid.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Intuition

The presence of heteroskedastic errors should not alter the central

position of the OLS line (unbiasedness).
OLS weigths all observations equally, but in this case it makes more
sense to pay more attention to observations where the variance is
smaller.
Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares
 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

The plan: what to do with heteroskedasticity.

1 Before abandoning OLS we will see how to test for
heteroskedasticity.
2 Strategy 1: Propose another more efficient and unbiased
estimator for (weighted least squares (WLS)) and a suitable
estimator for its variance.
3 Strategy 2: Keep using OLS (it is still unbiased, though
inefficient), but find a replacement for its variance (the old
one is biased under heteroskedasticity).

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Testing for heteroscedasticity

a) The White test

H0 : no heteroscedasticity, HA : there is heterocedasticity of some

form.
Consider a simple case with K = 3:

Yi = 1 + 2 X2i + 3 X3i + ui 1, . . . , n

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Steps to implement the test:

1 Estimate by OLS, save squared residuals in e2 .
2 Regress e2 on all variables, their squares and all possible
non-redundant cross-products. In our case, regress e2 on
1, X2 , X3 , X22 , X32 , X2 X3 , and obtain R2 in this auxiliar
regression.
3 Under H0 , nR2 2 (p). p = number of explanatory variables
in the auxiliar regression minus one.
4 Reject Ho if nR2 is too large.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Intuition: The auxiliar model can be seen as trying to model the

variance of the error term. If the R2 of this auxiliar regression were
high, then we could explain the behavior of the squared residuals,
providing evidence that they are not constant.

Caveats:
Valid for large samples.
Informative if we do not reject the null (no heterocedasticity).
When it rejects the null: there is heterocedasticity. But we do
not have any information regarding what causes
heterocedasticity. This will cause some trouble when trying to
construct a GLS estimator, for which we need to know in a
very specific way what causes heterocedasticity.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

b) The Breusch-Pagan/Godfrey/Koenker test

Mechanically very similar to Whites test. Checks if certain
variables cause heterocedasticity. Consider the following
heteroscedastic model:

Y = X + u, ui normal, with E(u) = 0 and

V (ui ) = h(1 + 2 Z2i + 3 Z3i + . . . + p Zpi )

where h( ) is any positive function with two derivatives.

When 2 = . . . = p = 0, V (ui ) = h(1 ), a constant!!
Then, homoscedasticity H0 : 2 = 3 = . . . = p = 0 ,and
HA : 2 6= 0 3 6= 0 . . . p 6= 0.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Steps to implement the test:

1 Estimate by OLS, and save squared residuals e2i .
2 Regresss e2i on the Zik variables, k = 2, . . . , p and get (ESS).
The test statistic is:
1
ESS 2 (p 1) 2 (p)
2
under H0 , asymptotically. We reject if it is too large.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Comments:
Intuition is as in the White test (a model for the variance).
By focusing on a particular group, if we reject the null we
have a better idea of what causes heterocedasticity.
Accepting the null does not mean there isnt heterocedasticity
(why?).
Also a large sample test.
Koenker (1980) has proposed to use nRA2 as a test, which is

still valid if errors are non-normal.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Estimation and inference under heteroscedasticity

For simplicity, consider the two variable case

Yi = 1 + 2 Xi + ui
where now the error term is heteroskedastic, that is

V (ui ) = i2 , i = 1, . . . , n
We will assume all the other classical assumptions still hold

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Divide each observation of the linear model by i :

Yi 1 Xi ui
= 1 + 2 +
i i i i
Yi = 1 X1i

+ 2 X2i
+ k Xki + ui

Note that V (ui ) = V (ui /i ) = 1, then the residuals of this

transformed model are homoscedastic.
Then, if we know i2 , the BLUE is simply the OLS estimator using
the transformed variables.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

In our case, the OLS with the transfored model is

Pn
xi yi
2,wls = Pi=1
n 2
i=1 xi

1,wls = Y 2,wls X

with Yi = Yi /i , Xi = Xi /i and lowercase letters are deviations

from sample means, as usual.

This is the weighted least squares estimator.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

The name weighted least squares come from the fact that the
estimator can be obtained by solving the following minimization
problem
n
X 1 2
min e
2 i
i=1 i

that is, errors enter the SSR weighted by the inverse of the
variance for each observation: we pay more attention to
observations with smaller variance.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Problem: in practice we do not know i2 . This leads to two

strategies
1 Use WLS: Pros: estimates will be unbiased and efficient.
Cons: we need to know the variances in advance (or make
assumptions)
2 Keep OLS but change its variance estimator: Think again
about the effects of heteroscedasticity on standard estimation
procedures. OLS is still unbiased though not efficient (not
that bad...). But, S 2 (X 0 X)1 is biased, which invalidates
inference (this is bad!). Then, a second strategy: keeping
OLS for and look for a valid estimator for its variance. Pros:
no assumptions needed, unbiased. Cons: we will lose
efficiency with respect to the WLS case (if available).

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

1) Known variance structure: WLS

Consider our simple two-variable case: Yi = 1 + 2 Xi + ui
Strategy: assume some particular forms of heteroscedasticity.
a) V (ui ) = 2 Xi2
2 is an unknown constant. Divide all observations by Xi

Yi 1 ui
= 1 + 2 +
Xi Xi Xi

Yi = 1 X0i

+ 2 + ui
X2
Note E(ui ) = E(ui /Xi )2 = 2 Xi2 = 2
i

Errors of the transformed model are homocedastic. Do OLS on the

transformed model!
Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares
 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

We do not need to know 2 . What have just divided by the

part of the standard error that varies over observations, that
is, by Xi .
This strategy provides a WLS.
Careful with interpretations. The intercept of the transformed
model is the slope of the original model and that the slope of
the transformed model is the intercept of the original one.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

b) V (u) = 2 Xi

Y 1 ui
i
p
= 1 + 2 Xi +
Xi Xi Xi

Yi = 1 X0i

+ 2 X1i + ui

As for implementation and interpretation, note that the

transformed model has no intercept, and that the coefficient of the
first explanatory variable corresponds to the intercept of the
original model, and the coefficient of the second variable
corresponds to the slope of the original model.
Problem with these strategies: it is difficult to find an exact form
for heterocedasticity

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

2) Unknown variance structure

Alternative strategy: retain OLS (still unbiased though not

efficient) and look for valid estimators for its variance. Variance
matrix of OLS under heteroscedasticity can be shown to be:

V (OLS ) = (X 0 X)1 X 0 X(X 0 X)1

= diag(12 , 22 , . . . , n2 ).

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

White (1980): a consistent estimator for X 0 X is X 0 DX,

D = diag(e21 , e22 , . . . , e2n ), ei s OLS residuals
Then, a heteroscedasticity consistent estimator of the variance
matrix is:

V (OLS )HC = (X 0 X)1 X 0 DX(X 0 X)1

Strategy: use OLS but replace S 2 (X 0 X)1 by Whites consistent

estimator. This strategy is not efficient, but it
does not require assumptions about the structure of
heteroscedasticity.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares

 OLS Under Heteroskedasticity
Testing for Heteroskedasticity

Summary
Heteroskedasticity makes OLS inefficient and invalidates the
standard estimator of its variance, and hence invalidates
inference (t tests, F tests, etc.).
The WLS estimator is efficient and unbiased but it depends on
knowing the variance structure. In practice is seldom available.
In practice it is more common to keep OLS and replace its
variance estimator by Whites consistent method, which does
not require any assumptions.

Walter Sosa-Escudero Heteroskedasticity and Weighted Least Squares