Multiple Regression

September 21, 2014 Msc. Ana Moraga Pumarino 1

What is a Multiple Regression?
A Multiple regression is concerned with the
relationship between a dependent or
explained variable and more than one
independent or explanatory variables.

September 21, 2014 Msc. Ana Moraga Pumarino 2
Multiple Regression
The consumption function:



Where
1
is the intercept
2
and
3
are slope
coefficients and u is the stochastic or random
error term.
September 21, 2014 Msc. Ana Moraga Pumarino 3
1 2 3 d
C Y r u
Multiple Regression

2
is the effect of disposable income (Y) on
consumption (C) holding (r) interest rates
constant.

3
is the effect of r on C holding Y constant.

September 21, 2014 Msc. Ana Moraga Pumarino 4
Estimation
To calculate OLS estimators

September 21, 2014 Msc. Ana Moraga Pumarino 5
2 2
1 1
minimize ( )
n n
i i
i i
e Y Y



2 2
0 1 1 2 2
1 1
( )
n n
i i
i i
e Y b b X b X



Estimation
To minimize the sum of squared errors we
differentiate the expression



with respect tob
0
, b
1
and b
2
.
September 21, 2014 Msc. Ana Moraga Pumarino 6
2 2
0 1 1 2 2
1 1
( )
n n
i i
i i
e Y b b X b X



Adjusted R square
R
2
is just the ratio of explained to total
variation.
It does not take into account the loss of degrees of
freedom as one adds new variables to the model.
Adjusted R
2
takes this factor into account
September 21, 2014 Msc. Ana Moraga Pumarino 7
Adjusted R square
The more variables you add to model the
higher the R
2
.
This does not mean that you should add as
many variables to you model as possible.
The number of variables in your model
should be guided by economic theory.
September 21, 2014 Msc. Ana Moraga Pumarino 8
Adjusted R square
2
/( )
1
/( 1)
RSS n k
R
TSS n

September 21, 2014 Msc. Ana Moraga Pumarino 9

2
1
RSS
R
TSS

Adjusted R
2

2
( 1)
1
( )
RSS n
R
TSS n k

September 21, 2014 Msc. Ana Moraga Pumarino 10

2
2
( 1)
1 (1 )
( )
n
R R
n k

En un Modelo de Regresin Lineal Mltiple,

los parmetros que acompaan a las variables
explicativas tambin se les denomina
coeficientes de regresin parcial, pues miden
el efecto sobre la variable explicada ante
cambios en la variable explicativa a la que
acompaan, cuando estn presentes otras
variables .
Se interpreta como el cambio en la variable
explicada producido por un cambio unitario
en la variable explicativa, manteniendo
constantes las dems variables.
September 21, 2014 Msc. Ana Moraga Pumarino 11
Los coeficientes de regresin parcial
dependen de las unidades de medida de las
variables a las que acompaan, por lo que no
son directamente comparables.
Para una mejor interpretacin de la
estimacin de un modelo de regresin, es
necesario observar e interpretar los
estimadores de los coeficientes
estandarizados. Los coeficientes
estandarizados, as como las elasticidades, no
dependen de las unidades de medida de las
variables a las que acompaan.

September 21, 2014 Msc. Ana Moraga Pumarino 12
Los coeficientes beta estandarizados se
obtienen a travs de la multiplicacin del
coeficiente por la desviacin estndar de la
variable independiente y dividiendo por la
desviacin estndar de la variable
dependiente. As se puede determinar cual es
la variable explicativa que tiene mayor peso
para la explicacin de la variable dependiente.

September 21, 2014 Msc. Ana Moraga Pumarino 13
Hypothesis Testing
Hypothesis testing of individual coefficients is
the same as hypothesis testing in simple
regression.
Hypotheses are tested using individual t
statistics.



September 21, 2014 Msc. Ana Moraga Pumarino 14
Hypothesis Testing
The only difference with multiple regression is
in degrees of freedom.
We now have (n-k) degrees of freedom where
k is the number of parameters estimated in
the model


September 21, 2014 Msc. Ana Moraga Pumarino 15
Hypothesis Testing
2 2
2
( )
b
t
se b

September 21, 2014 Msc. Ana Moraga Pumarino 16

Hypothesis Testing
Testing the hypothesis that the slope
coefficients are all simultaneously equal to
zero is the same as testing the hypothesis : H
0
:
R
2
= 0.

This is a test of the overall significance of the
model.
September 21, 2014 Msc. Ana Moraga Pumarino 17
Hypothesis Testing
TSS = ESS + RSS
(n-1) = (k-1) + (n-k)
Total SS = Model SS + Error SS

September 21, 2014 Msc. Ana Moraga Pumarino 18
Hypothesis Testing
The overall significance of the model is
determined by an F test

September 21, 2014 Msc. Ana Moraga Pumarino 19
( 1)
( )
ESS
K
F
RSS
N K

Hypothesis Testing
The F statistic is the ratio of the explained
variation to the unexplained variation.
In order for a model to be significant the
explained variation must be larger than the
unexplained variation.
September 21, 2014 Msc. Ana Moraga Pumarino 20
Hypothesis Testing
There is a relationship between R
2
and F

September 21, 2014 Msc. Ana Moraga Pumarino 21
2
2
( 1)
(1 ) ( )
R k
F
R n k

Specification Error
One assumption of the classical linear model
is that the model is correctly specified.
Must have the right variables
Must have the correct functional form
September 21, 2014 Msc. Ana Moraga Pumarino 22
Specification Error
This assumption is critical if we are to obtain
unbiased estimators.
An estimator is unbiased if:
September 21, 2014 Msc. Ana Moraga Pumarino 23
2 2
( ) E

Specification Error
Suppose we have the following model:
September 21, 2014 Msc. Ana Moraga Pumarino 24
0 1 1 2 2 i i
Y X X e
Specification Error
However we estimate:
September 21, 2014 Msc. Ana Moraga Pumarino 25
0 1 1 i i
Y X e
Specification Error
In this case we do not have an unbiased
estimator:


Where
September 21, 2014 Msc. Ana Moraga Pumarino 26
1 1 2 12 i
b e
2 0 12 1 i
X b b X u
Specification Error
In this case:
September 21, 2014 Msc. Ana Moraga Pumarino 27
1 1 2 12
( ) E b