2.

1 An Economic Model
2.2 An Econometric Model
2.3 Estimating the Regression Parameters
2.4 Assessing the Least Squares Estimators
2.5 The Gauss-Markov Theorem
2.6 The Probability Distributions of the Least
Squares Estimators
2.7 Estimating the Variance of the Error Term
Figure 2.1a Probability distribution of food expenditure y given income x = $1000

Slide 2-3
Figure 2.1b Probability distributions of food expenditures y
given incomes x = $1000 and x = $2000

Slide 2-4
 The simple regression function
Figure 2.2 The economic model: a linear relationship between
average per person food expenditure and income

Slide 2-6
 Slope of regression line

denotes change in
 Figure 2.3 The probability density function for y at two levels of income

Principles of Econometrics, 3rd Edition Slide 2-8

 E y | x 1 2 x

Principles of Econometrics, 3rd Edition Slide 2-9

 var y | x 2

Principles of Econometrics, 3rd Edition Slide 2-10

 cov yi , y j 0

Principles of Econometrics, 3rd Edition Slide 2-11

Principles of Econometrics, 3rd Edition Slide 2-12
 y : N 1 2 x, 2

Principles of Econometrics, 3rd Edition Slide 2-13

Principles of Econometrics, 3rd Edition Slide 2-14
 2.2.1 Introducing the Error Term
The random error term is defined as

Rearranging gives

y is dependent variable; x is independent variable

Principles of Econometrics, 3rd Edition Slide 2-15

 The expected value of the error term, given x, is

E e | x E y | x 1 2 x 0

The mean value of the error term, given x, is zero.

Principles of Econometrics, 3rd Edition Slide 2-16

 Figure 2.4 Probability density functions for e and y

Principles of Econometrics, 3rd Edition Slide 2-17

 y 1 2 x e

Principles of Econometrics, 3rd Edition Slide 2-18

 E (e) 0

E ( y ) 1 2 x

Principles of Econometrics, 3rd Edition Slide 2-19

 var(e) var( y )
2

Principles of Econometrics, 3rd Edition Slide 2-20

 cov(ei , e j ) cov( yi , y j ) 0

Principles of Econometrics, 3rd Edition Slide 2-21

Principles of Econometrics, 3rd Edition Slide 2-22
 e : N 0, 2

Principles of Econometrics, 3rd Edition Slide 2-23

Principles of Econometrics, 3rd Edition Slide 2-24
 Figure 2.5 The relationship among y, e and the true regression line

Principles of Econometrics, 3rd Edition Slide 2-25

Principles of Econometrics, 3rd Edition Slide 2-26
 Figure 2.6 Data for food expenditure example

Principles of Econometrics, 3rd Edition Slide 2-27

 2.3.1 The Least Squares Principle
The fitted regression line is

The least squares residual

Principles of Econometrics, 3rd Edition Slide 2-28

 Figure 2.7 The relationship among y, and the fitted regression line

Principles of Econometrics, 3rd Edition Slide 2-29

 Any other fitted line

yi* b1* b2* xi

Least squares line has smaller sum of squared residuals

N N
if SSE = ei2 and SSE * = ei*2 then SSE < SSE *
i 1 i 1

Principles of Econometrics, 3rd Edition Slide 2-30

 Least squares estimates for the unknown parameters 1
and 2 are obtained my minimizing the sum of squares
function
N
S 1 , 2 ( yi 1 2 xi ) 2
i 1

Principles of Econometrics, 3rd Edition Slide 2-31

 The Least Squares Estimators

Principles of Econometrics, 3rd Edition Slide 2-32

 2.3.2 Estimates for the Food Expenditure Function

b2
x x y y 18671.2684
i
i
10.2096
x x
2
i
1828.7876

b1 y b2 x 283.5735 (10.2096)(19.6048) 83.4160

A convenient way to report the values for b1 and b2 is to

write out the estimated or fitted regression line:

yi 83.42 10.21xi
Principles of Econometrics, 3rd Edition Slide 2-33
 Figure 2.8 The fitted regression line

Principles of Econometrics, 3rd Edition Slide 2-34

 2.3.3 Interpreting the Estimates

The value b2 = 10.21 is an estimate of 2, the amount by

which weekly expenditure on food per household increases
when household weekly income increases by $100. Thus, we
estimate that if income goes up by $100, expected weekly
expenditure on food will increase by approximately $10.21.

Strictly speaking, the intercept estimate b1 = 83.42 is an

estimate of the weekly food expenditure on food for a
household with zero income.

Principles of Econometrics, 3rd Edition Slide 2-35

 2.3.3a Elasticities
Income elasticity is a useful way to characterize the responsiveness
of consumer expenditure to changes in income. The elasticity of a
variable y with respect to another variable x is

percentage change in y y y y x

percentage change in x x x x y
In the linear economic model given by (2.1) we have shown that

E y
2
x
Principles of Econometrics, 3rd Edition Slide 2-36
 The elasticity of mean expenditure with respect to income is

A frequently used alternative is to calculate the elasticity at the

point of the means because it is a representative point on the
regression line.

x 19.60
b2 10.21 .71
y 283.57

Principles of Econometrics, 3rd Edition Slide 2-37

 2.3.3b Prediction
Suppose that we wanted to predict weekly food expenditure for a
household with a weekly income of $2000. This prediction is
carried out by substituting x = 20 into our estimated equation to
obtain

yi 83.42 10.21xi 83.42 10.21(20) 287.61

We predict that a household with a weekly income of $2000 will
spend $287.61 per week on food.

Principles of Econometrics, 3rd Edition Slide 2-38

 2.3.3c Examining Computer Output

Figure 2.9 EViews Regression Output

Principles of Econometrics, 3rd Edition Slide 2-39

 2.3.4 Other Economic Models
The log-log model

ln( y ) 1 2 ln( x)

d [ln( y )] 1 dy

dx y dx

d [1 2 ln( x )] 1
2
dx x
dy x
2
dx y
Principles of Econometrics, 3rd Edition Slide 2-40
 2.4.1 The estimator b2

N
b2 wi yi
i 1

xi x
wi
( xi x )2

b2 2 wi ei

Principles of Econometrics, 3rd Edition Slide 2-41

 2.4.2 The Expected Values of b1 and b2
We will show that if our model assumptions hold, then E b , which means
2 2
that the estimator is unbiased.
We can find the expected value of b2 using the fact that the expected value of a sum
is the sum of expected values

E (b2 ) E 2 wi ei E 2 w1e1 w2 e2 L wN eN

E 2 E w1e1 E w2 e2 L E wN eN

E (2 ) E ( wi ei )

2 wi E (ei ) 2
using and
E wi ei wi E ei E (ei ) 0
Principles of Econometrics, 3rd Edition Slide 2-42
 2.4.3 Repeated Sampling

Principles of Econometrics, 3rd Edition Slide 2-43

 The variance of b2 is defined as var b2 E b2 E b2
2

Figure 2.10 Two possible probability density functions for b2

Principles of Econometrics, 3rd Edition Slide 2-44

 2.4.4 The Variances and Covariances of b1 and b2

If the regression model assumptions SR1-SR5 are correct (assumption SR6 is not
required), then the variances and covariance of b1 and b2 are:

var(b1 ) 2
xi2

N ( xi x ) 2

2
var(b2 )
( xi x )2
x
cov(b1 , b2 ) 2 2
i
( x x )

Principles of Econometrics, 3rd Edition Slide 2-45

 2.4.4 The Variances and Covariances of b1 and b2
The larger the variance term 2, the greater the uncertainty there is in the
statistical model, and the larger the variances and covariance of the least squares
estimators.
The larger the sum of squares, ( x x ) 2, the smaller the variances of the least
i
squares estimators and the more precisely we can estimate the unknown
parameters.
The larger the sample size N, the smaller the variances and covariance of the
least squares estimators.
The larger this term 2 is, the larger the variance of the least squares estimator
b1. ix
The absolute magnitude of the covariance increases the larger in magnitude is
the sample mean , and the covariance has a sign opposite to that of .
x x

Principles of Econometrics, 3rd Edition Slide 2-46

 The variance of b2 is defined as var b2 E b2 E b2
2

Figure 2.11 The influence of variation in the explanatory variable x on precision of estimation
(a) Low x variation, low precision (b) High x variation, high precision

Principles of Econometrics, 3rd Edition Slide 2-47

Principles of Econometrics, 3rd Edition Slide 2-48
 1. The estimators b1 and b2 are best when compared to similar estimators, those
which are linear and unbiased. The Theorem does not say that b1 and b2 are the
best of all possible estimators.
2. The estimators b1 and b2 are best within their class because they have the
minimum variance. When comparing two linear and unbiased estimators, we
always want to use the one with the smaller variance, since that estimation rule
gives us the higher probability of obtaining an estimate that is close to the true
parameter value.
3. In order for the Gauss-Markov Theorem to hold, assumptions SR1-SR5 must
be true. If any of these assumptions are not true, then b1 and b2 are not the best
linear unbiased estimators of 1 and 2.

Principles of Econometrics, 3rd Edition Slide 2-49

 4. The Gauss-Markov Theorem does not depend on the assumption of normality
(assumption SR6).
5. In the simple linear regression model, if we want to use a linear and unbiased
estimator, then we have to do no more searching. The estimators b1 and b2 are
the ones to use. This explains why we are studying these estimators and why
they are so widely used in research, not only in economics but in all social and
physical sciences as well.
6. The Gauss-Markov theorem applies to the least squares estimators. It does not
apply to the least squares estimates from a single sample.

Principles of Econometrics, 3rd Edition Slide 2-50

 If we make the normality assumption (assumption SR6 about the error term)
then the least squares estimators are normally distributed

2 xi2
b1 ~ N 1 , 2
N i
( x x )

2
b2 ~ N 2 , 2
i
( x x )

Principles of Econometrics, 3rd Edition Slide 2-51

 The variance of the random error ei is

var(ei ) 2 E[ei E (ei )]2 E (ei2 )

if the assumption E(ei) = 0 is correct.
Since the expectation is an average value we might consider estimating 2 as the
average of the squared errors,
2 i
2
e
N
Recall that the random errors are

ei yi 1 2 xi

Principles of Econometrics, 3rd Edition Slide 2-52

 The least squares residuals are obtained by replacing the unknown parameters by their
least squares estimates,

ei yi yi yi b1 b2 xi

2

ei
2

N
There is a simple modification that produces an unbiased estimator, and that is

2
i
e 2

N 2

E ( 2 ) 2

Principles of Econometrics, 3rd Edition Slide 2-53

 Replace the unknown error variance in (2.14)-(2.16) by
2 2
to obtain:

b
var 2 xi2

N ( xi x ) 2
1

b
2
var
i
2
( x x ) 2

x
cov b1 , b2
2
2
( xi x )

Principles of Econometrics, 3rd Edition Slide 2-54

 The square roots of the estimated variances are the standard errors of b1
and b2.

b
se b1 var 1

b
se b2 var 2

Principles of Econometrics, 3rd Edition Slide 2-55


2
ei
2

304505.2
8013.29
N 2 38

Principles of Econometrics, 3rd Edition Slide 2-56

 The estimated variances and covariances for a regression are arrayed
in a rectangular array, or matrix, with variances on the diagonal and
covariances in the off-diagonal positions.

b
var b , b
cov
1 1 2

b ,b
cov b
var
1 2 2

Principles of Econometrics, 3rd Edition Slide 2-57

 For the food expenditure data the estimated covariance matrix is:

C INCOME
C 1884.442 -85.90316
INCOME -85.90316 4.381752

Principles of Econometrics, 3rd Edition Slide 2-58

 b 1884.442
var 1

b 4.381752
var 2

b , b 85.90316
cov 1 2

b 1884.442 43.410
se b1 var 1

b 4.381752 2.093
se b2 var 2

Principles of Econometrics, 3rd Edition Slide 2-59

Principles of Econometrics, 3rd Edition Slide 2-60
Principles of Econometrics, 3rd Edition Slide 2-61
 (2A.1)

(2A.2)

Principles of Econometrics, 3rd Edition Slide 2-62

 Figure 2A.1 The sum of squares function and the minimizing values b1 and b2

Principles of Econometrics, 3rd Edition Slide 2-63

 (2A.3)

(2A.4)

(2A.5)

Principles of Econometrics, 3rd Edition Slide 2-64

 1
i
( x x ) 2
x 2x i
x
2
i N x x 2 x
2
N i
2
ix N x 2

N (2B.1)

xi2 2 N x 2 N x 2 xi2 N x 2

xi
2

( xi x ) 2
x N x x x xi x
2
i
2 2
i
2
i
N
(2B.2)

xi yi
( xi x )( yi y ) xi yi N x y xi yi N
(2B.3)

Principles of Econometrics, 3rd Edition Slide 2-65

 We can rewrite b2 in deviation from the mean form as:

b2
( xi x )( yi y )
( xi x )2

Principles of Econometrics, 3rd Edition Slide 2-66

 xi x 0

b2
( xi x )( yi y ) ( xi x ) yi y ( xi x )

( xi x ) 2
( xi x )2

( xi x ) y i

( xi x )
yi wi yi
2
i( x x ) 2
i
( x x )

Principles of Econometrics, 3rd Edition Slide 2-67

 To obtain (2.12) replace yi in (2.11) by yi 1 2 xi ei and simplify:

b2 wi yi wi (1 2 xi ei )

1 wi 2 wi xi wi ei

2 wi ei

Principles of Econometrics, 3rd Edition Slide 2-68


xi x 1
wi xi x 0
xi x xi x
2 2

wi xi 1
2 wi xi 2

( xi x ) 0

Principles of Econometrics, 3rd Edition Slide 2-69

 xi x xi x xi x
2

xi x xi x xi x

xi x xi

xi x xi xi x xi
wi xi 1
xi x xi x xi
2

Principles of Econometrics, 3rd Edition Slide 2-70

 b2 2 wi ei

var b2 E b2 E b2
2

Principles of Econometrics, 3rd Edition Slide 2-71

 var b2 E 2 wi ei 2
2

wi e i
2
E

E i i i j i j
w 2 2
e 2 w w e e [square of bracketed term]
i j

wi2 E ei2 2 wi w j E ei e j [because wi not random]

i j

2 wi2

2

xi x
2

Principles of Econometrics, 3rd Edition Slide 2-72

 2 var ei E ei E ei E ei 0 E ei2
2 2

cov ei , e j E ei E ei e j E e j
E ei e j 0

xi x xi x
2 2
1
wi
2
2 2

xi x xi x xi x
2 2 2

var aX bY a 2 var X b 2 var Y 2ab cov X , Y

Principles of Econometrics, 3rd Edition Slide 2-73

 var b2 var 2 wi ei [since 2 is a constant]

= wi2 var ei wi w j cov ei , e j [generalizing the variance rule]

i j

= wi2 var ei [using cov ei , e j 0]

2 wi2 [using var ei 2 ]

2

xi x
2

Principles of Econometrics, 3rd Edition Slide 2-74

 Let b2* ki yi be any other linear estimator of 2.

Suppose that ki = wi + ci.

b2* ki yi ( wi ci ) yi ( wi ci )(1 2 xi ei )

( wi ci ) 1 ( wi ci ) 2 xi ( wi ci ) ei
(2F.1)
1 wi 1 ci 2 wi xi 2 ci xi ( wi ci ) ei

1 ci 2 2 ci xi ( wi ci ) ei

Principles of Econometrics, 3rd Edition Slide 2-75

 E (b2* ) 1 ci 2 2 ci xi ( wi ci )E (ei )
(2F.2)
1 ci 2 2 ci xi

ci 0 and ci xi 0 (2F.3)

b2* ki yi 2 ( wi ci ) ei (2F.4)

Principles of Econometrics, 3rd Edition Slide 2-76

 ci xi x 1 x
ci wi 2 ci xi ci 0
xi x i xi x
2 2
x x

var b2* var wi ci e i wi ci var ei

2
2

2 wi ci 2 wi2 2 ci2
2

var b2 2 ci2

var b2

Principles of Econometrics, 3rd Edition Slide 2-77