ECOM30001

Basic Econometrics
Semester 1, 2017

Week 1, Lecture 2
Basic Linear Model: Assumptions
Reading: Hill et. al Chapter 5

1 The Linear Regression Model

Introduction
The Error Term
Assumptions about the Error Term

2 The Least Squares Principle

3 Properties of the OLS Residuals

Introduction

(economic) theory describes the average behaviour of many

individuals:
identifies relationships between economic variables
makes predictions about the direction of outcomes when a
variable is altered
we are ultimately interested in whether one variable (x)
causes another variable (y ), all else equal
problem: How can we uncover the magnitude of these causal
relationships without experimental data?
technique use the regression model with appropriate
assumptions

Introduction

The regression model relates the dependent variable (y ) to a

number of explanatory variables (X = x1 , x2 , . . . xK ) through the
linear (in parameters) equation:

yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i

where 0 , 1 , . . . K are unknown parameters

econometrics recognizes that actual behaviour depends upon
both average behaviour and a random component
econometrics uses a sample of data on (y , X) to learn about
the magnitude of the unknown parameters 0 , 1 , . . . K
the random error can never be known because the
population parameters 0 , 1 , . . . K governing the
relationship between y and X are unknown
The Error Term

The error term represents:

any other factors other than X that affect y and are not
included in the model.
approximation error arising from the assumed linear functional
form.
unpredictable random behaviourknowledge of all variables
that affect y might not be sufficient to perfectly predict y .

The Economic Model

yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i

1 the intercept 0 represents the average value of y when all the

X s are zero
2 the slope parameter j represents the expected change in y
associated with a unit change in Xj , all else constant

E [y |X] 
j = for j = 1, 2, . . . K
Xj all other X s constant
Assumptions about the Error Term

The econometric model:

yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i

ultimately we want to answer questions such as:

What is the effect of X2 on y , holding all other
factors fixed (X1 , X3 , . . . XK )?

but we have grouped all these other factors into the error
term .
to answer these questions we need to make assumptions
about the error term and the relationship between the error
term and the X s.

Assumption 1 E [|x] = 0

the critical assumption is the error term has an expected value

of 0, given any value of the X s
This means that knowledge of the X s does not change the
expected value of the random error .
the average of the omitted other factors does not depend
upon the X s
this does not mean that, in any sample of data, the errors
average out
the (conditional) expectation is taken over all possible
realizations of the error
Assumption 1 E [|x] = 0

Given:
yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i
the assumption E [i |Xi ] = 0 implies

E [yi |Xi ] = 0 + 1 X1i + 2 X2i + . . . K XKi

the other factors do not systematically affect average behaviour

yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i

= E [yi |Xi ] + i

The econometric model decomposes y into

1 a systematic component of y explained by X
2 a random component of y not explained by X

Assumptions
MR1: The correct model is:

yi = 0 + 1 X1i + 2 X2i + . . . K XKi + i

MR2: The conditional expected value of the random errors is

zero:
E [i |Xi ] = 0

MR3 Homoskedasticity : The variance of the random errors is

constant and independent of the X s:

VAR[i |Xi ] = 2

MR4: any pair of random errors are uncorrelated:

COV(i , j |Xi , Xj ) = 0 for all i, j = 1, 2, . . . N, i 6= j

Assumptions about the Explanatory Variables

1 MR5a: the explanatory variables are not-randomthe values

of all of the X s are known prior to observing the (realized)
values of the dependent variable.
fixed in repeated sampling assumption.
we will relax this assumption later in the subject.
2 MR5b: any one of the X s is not an exact linear function of
any of the other X s. This is equivalent to assuming that
none of the X s are redundant
if this second assumption is violated, we have a condition
called exact collinearity and the least squares procedure will
failmore on this later

The Least Squares Principle

the Least Squares Principle estimates (0 , 1 , . . . K ) such

that the squared difference between the fitted line and the
observed value of y is minimized.
let (b0 , b1 , . . . bK ) be estimates of the unknown parameters
(0 , 1 , . . . K )
define the fitted line:

ybi = b0 + b1 X1i + b2 X2i + . . . + bK XKi

and define the least squares residuals

ebi = (yi ybi ) = yi (b0 + b1 X1i + b2 X2i + . . . + bK XKi )

The Least Squares Principle

Why might we want to minimize the squared difference

between the fitted line and the observed value of y ?
P
Why not minimize ebi the minimum value would be at
so the Pfitted line could be set arbitrarily high to
minimize ebi
P
Why not set fitted line so ebi = 0 a large positive value of
ebi > 0 would cancel out a large negative value ebi < 0. In
addition any fitted line passing through the (sample) means of
y and x would satisfy this criteria (infinite number of
potential fitted lines)

The Least Squares Principle

choose (0 , 1 , . . . K ) to minimize the sum of squared errors:

N
X
min S= [yi 0 1 X1i . . . K XKi ]2
{0 ,1 ...,K }
i=1

first order conditions:

N
S X
= 2 [yi 0 1 X2i . . . K XKi ] = 0
0
i=1

for j = 1, 2, . . . K
N
S X
= 2 [yi 0 1 X1i . . . K xKi ] Xji = 0
j
i=1
The Least Squares Principle

let (b0 , b1 , . . . bK ) be the values of (0 , 1 , . . . K ) that

minimize the sum of squares function S:
 N
S  X
= 2 [yi b0 b1 X1i . . . bK XKi ] = 0
0 b1 ,b2 ,...bK
i=1

for j = 1, 2, . . . K
 T
S  X
= 2 [yi b0 b1 X1i . . . bK XKi ] Xji = 0
j b0 ,b1 ,...bK
i=1

we have (K + 1) linear equations in (K + 1) unknowns

Objective

Our objective is to use a random sample of data on y and X

and the econometric model to learn about the values of the
population parameters (0 , 1 , . . . K )
we use the Ordinary Least Squares estimators for the
population parameters.
the expressions for (b0 , b1 , . . . bK ) are estimators
they are formulas that allow us to find the values of
(b0 , b1 , . . . bK ) no matter what the values of y and the X s
turn out to bethe formulas will be the same for different
samples of data.
(b0 , b1 , . . . bK ) are random variablestheir values depend
upon the sample data y and X.
when the sample data are substituted into the formulas we
obtain numbers that are the observed values of random
variables. We call these least squares estimates
Properties of the OLS Residuals
1 when there is an intercept tern:
N
X
ebi = 0
i=1

2 for each j = 1, 2, . . . K :
N
X N
X N
X
ebi X1i = 0 and ebi X2i = 0 and . . . ebi XKi = 0
i=1 i=1 i=1

3 these two properties imply:

X X
ebi ybi = ebi [b0 + b1 X1i + . . . bK XKi ]
X X X
= b0 ebi + b1 ebi X1i + . . . bK ebi XKi
=0