The Rules of Summation

n
i=1
x
i
= x
1
x
2
x
n

n
i=1
a = na

n
i=1
ax
i
= a
n
i=1
x
i

n
i=1
(x
i
y
i
) =
n
i=1
x
i

n
i=1
y
i

n
i=1
(ax
i
by
i
) = a
n
i=1
x
i
b
n
i=1
y
i

n
i=1
(a bx
i
) = na b
n
i=1
x
i
x =

n
i=1
x
i
n
=
x
1
x
2
x
n
n

n
i=1
(x
i
x) = 0

2
i=1

3
j=1
f (x
i
; y
j
) =
2
i=1
f (x
i
; y
1
) f (x
i
; y
2
) f (x
i
; y
3
) [ [
= f (x
1
; y
1
) f (x
1
; y
2
) f (x
1
; y
3
)
f (x
2
; y
1
) f (x
2
; y
2
) f (x
2
; y
3
)
Expected Values & Variances
E(X) = x
1
f (x
1
) x
2
f (x
2
) x
n
f (x
n
)
=
n
i=1
x
i
f (x
i
) =
x
x f (x)
E g(X) [ [ =
x
g(x) f (x)
E g
1
(X) g
2
(X) [ [ =
x
g
1
(x) g
2
(x) [ [ f (x)
=
x
g
1
(x) f (x)
x
g
2
(x) f (x)
= E g
1
(X) [ [ E g
2
(X) [ [
E(c) = c
E(cX) = cE(X)
E(a cX) = a cE(X)
var(X) = s
2
= E[X E(X)]
2
= E(X
2
) [E(X)]
2
var(a cX) = E[(a cX) E(a cX)]
2
= c
2
var(X)
Marginal and Conditional Distributions
f (x) =
y
f (x; y) for each value X can take
f (y) =
x
f (x; y) for each value Y can take
f (x[y) = P X = x[Y = y [ [ =
f (x; y)
f (y)
If X and Y are independent random variables, then
f (x,y) = f (x)f ( y) for each and every pair of values
x and y. The converse is also true.
If X and Y are independent random variables, then the
conditional probability density function of X given that
Y = y is f (x[y) =
f (x; y)
f (y)
=
f (x) f (y)
f (y)
= f (x)
for each and everypair of values x and y. The converse is
also true.
Expectations, Variances & Covariances
cov(X; Y) = E[(XE[X[)(YE[Y[)[
=
x

y
x E(X) [ [ y E(Y) [ [ f (x; y)
r =
cov(X;Y)

var(X)var(Y)
_
E(c
1
X c
2
Y) = c
1
E(X) c
2
E(Y)
E(X Y) = E(X) E(Y)
var(aXbYcZ) =a
2
var(X) b
2
var(Y) c
2
var(Z)
2abcov(X,Y) 2accov(X,Z) 2bccov(Y,Z)
If X, Y, and Z are independent, or uncorrelated, random
variables, then the covariance terms are zero and:
var(aX bY cZ) = a
2
var(X)
b
2
var(Y) c
2
var(Z)
Normal Probabilities
If X - N(m, s
2
), then Z =
X m
s
-N(0; 1)
If X - N(m, s
2
) and a is a constant, then
P(X _ a) = P Z _
a m
s
_ _
If X -N(m; s
2
) and a and b are constants; then
P(a _ X _ b) = P
am
s
_ Z _
b m
s
_ _
Assumptions of the Simple Linear Regression
Model
SR1 The value of y, for each value of x, is y = b
1

b
2
x e
SR2 The average value of the random error e is
E(e) = 0 since we assume that E(y) = b
1
b
2
x
SR3 The variance of the random error e is var(e) =
s
2
= var(y)
SR4 The covariance between any pair of random
errors, e
i
and e
j
is cov(e
i
, e
j
) = cov(y
i
, y
j
) = 0
SR5 The variable x is not random and must take at
least two different values.
SR6 (optional ) The values of e are normally dis-
tributed about their mean e - N(0, s
2
)
Least Squares Estimation
If b
1
and b
2
are the least squares estimates, then
^y
i
= b
1
b
2
x
i
^e
i
= y
i
^y
i
= y
i
b
1
b
2
x
i
The Normal Equations
Nb
1
Sx
i
b
2
=Sy
i
Sx
i
b
1
Sx
2
i
b
2
= Sx
i
y
i
Least Squares Estimators
b
2
=
S(x
i
x)(y
i
y)
S(x
i
x)
2
b
1
= y b
2
x
Elasticity
h =
percentage change in y
percentage change in x
=
Dy=y
Dx=x
=
Dy
Dx

x
y
h =
DE(y)=E(y)
Dx=x
=
DE(y)
Dx

x
E(y)
= b
2

x
E(y)
Least Squares Expressions Useful for Theory
b
2
= b
2
Sw
i
e
i
w
i
=
x
i
x
S(x
i
x)
2
Sw
i
= 0; Sw
i
x
i
= 1; Sw
2
i
= 1=S(x
i
x)
2
Properties of the Least Squares Estimators
var(b
1
) = s
2
Sx
2
i
NS(x
i
x)
2
_ _
var(b
2
) =
s
2
S(x
i
x)
2
cov(b
1
; b
2
) = s
2
x
S(x
i
x)
2
_ _
Gauss-Markov Theorem: Under the assumptions
SR1SR5 of the linear regression model the estimators
b
1
and b
2
have the smallest variance of all linear and
unbiased estimators of b
1
and b
2
. They are the Best
Linear Unbiased Estimators (BLUE) of b
1
and b
2
.
If we make the normality assumption, assumption
SR6, about the error term, then the least squares esti-
mators are normally distributed.
b
1
- N b
1
;
s
2
x
2
i
NS(x
i
x)
2
_ _
; b
2
- N b
2
;
s
2
S(x
i
x)
2
_ _
Estimated Error Variance
^ s
2
=
S^e
2
i
N 2
Estimator Standard Errors
se(b
1
) =

var(b
1
)
_
; se(b
2
) =

var(b
2
)
_
t-distribution
If assumptions SR1SR6 of the simple linear regression
model hold, then
t =
b
k
b
k
se(b
k
)
- t
(N2)
; k = 1; 2
Interval Estimates
P[b
2
t
c
se(b
2
) _ b
2
_ b
2
t
c
se(b
2
)] = 1 a
Hypothesis Testing
Components of Hypothesis Tests
1. A null hypothesis, H
0
2. An alternative hypothesis, H
1
3. A test statistic
4. A rejection region
5. A conclusion
If the null hypothesis H
0
: b
2
= c is true, then
t =
b
2
c
se(b
2
)
- t
(N2)
Rejection rule for a two-tail test: If the value of the
test statistic falls in the rejection region, either tail of
the t-distribution, then we reject the null hypothesis
and accept the alternative.
Type I error: The null hypothesis is true and we decide
to reject it.
Type II error: The null hypothesis is false and we decide
not to reject it.
p-value rejection rule: When the p-value of a hypoth-
esis test is smaller than the chosen value of a, then the
test procedure leads to rejection of the null hypothesis.
Prediction
y
0
= b
1
b
2
x
0
e
0
; ^y
0
= b
1
b
2
x
0
; f = ^y
0
y
0

var( f ) = ^ s
2
1
1
N

(x
0
x)
2
S(x
i
x)
2
_ _
; se( f ) =

var( f )
_
A (1 a) 100% condence interval, or prediction
interval, for y
0
^y
0
t
c
se( f )
Goodness of Fit
S(y
i
y)
2
= S(^y
i
y)
2
S^e
2
i
SST = SSR SSE
R
2
=
SSR
SST
= 1
SSE
SST
= (corr(y; ^y))
2
Log-Linear Model
ln(y) = b
1
b
2
x e;

ln( y) = b
1
b
2
x
100 b
2
~ % change in y given a one-unit change in x:
^y
n
= exp(b
1
b
2
x)
^y
c
= exp(b
1
b
2
x)exp(^ s
2
=2)
Prediction interval:
exp

ln(y) t
c
se( f )
_ _
; exp

ln( y) t
c
se( f )
_ _
Generalized goodness-of-t measure R
2
g
=(corr(y;^y
n
))
2
Assumptions of the Multiple Regression Model
MR1 y
i
= b
1
b
2
x
i2
b
K
x
iK
e
i
MR2 E(y
i
) =b
1
b
2
x
i2
b
K
x
iK
=E(e
i
) = 0.
MR3 var(y
i
) = var(e
i
) = s
2
MR4 cov(y
i
, y
j
) = cov(e
i
, e
j
) = 0
MR5 The values of x
ik
are not random and are not
exact linear functions of the other explanatory
variables.
MR6 y
i
- N[(b
1
b
2
x
i2
b
K
x
iK
); s
2
[
=e
i
- N(0; s
2
)
Least Squares Estimates in MR Model
Least squares estimates b
1
, b
2
, . . . , b
K
minimize
S(b
1
, b
2
, . . . , b
K
) = (y
i
b
1
b
2
x
i2
b
K
x
iK
)
2
Estimated Error Variance and Estimator
Standard Errors
^ s
2
=
^e
2
i
N K
se(b
k
) =

var(b
k
)
_
Hypothesis Tests and Interval Estimates for Single Parameters
Use t-distribution t =
b
k
b
k
se(b
k
)
- t
(NK)
t-test for More than One Parameter
H
0
: b
2
cb
3
= a
When H
0
is true t =
b
2
cb
3
a
se(b
2
cb
3
)
- t
(NK)
se(b
2
cb
3
) =

var(b
2
) c
2
var(b
3
) 2c

cov(b
2
; b
3
)
_
Joint F-tests
To test J joint hypotheses,
F =
(SSE
R
SSE
U
)=J
SSE
U
=(N K)
To test the overall signicance of the model the null and alternative
hypotheses and F statistic are
H
0
: b
2
= 0; b
3
= 0; : : : ; b
K
= 0
H
1
: at least one of the b
k
is nonzero
F =
(SST SSE)=(K 1)
SSE=(N K)
RESET: A Specication Test
y
i
=b
1
b
2
x
i2
b
3
x
i3
e
i
^y
i
=b
1
b
2
x
i2
b
3
x
i3
y
i
=b
1
b
2
x
i2
b
3
x
i3
g
1
^y
2
i
e
i
; H
0
: g
1
=0
y
i
=b
1
b
2
x
i2
b
3
x
i3
g
1
^y
2
i
g
2
^y
3
i
e
i
; H
0
: g
1
=g
2
=0
Model Selection
AIC = ln(SSE=N) 2K=N
SC = ln(SSE=N) Kln(N)=N
Collinearity and Omitted Variables
y
i
= b
1
b
2
x
i2
b
3
x
i3
e
i
var(b
2
) =
s
2
(1 r
2
23
) (x
i2
x
2
)
2
When x
3
is omitted; bias(b
+
2
) = E(b
+
2
) b
2
= b
3

cov(x
2
; x
3
)

var(x
2
)
Heteroskedasticity
var(y
i
) = var(e
i
) = s
i
2
General variance function
s
2
i
= exp(a
1
a
2
z
i2
a
S
z
iS
)
Breusch-Pagan and White Tests for H
0
: a
2
= a
3
= = a
S
= 0
When H
0
is true x
2
= N R
2
- x
2
(S1)
Goldfeld-Quandt test for H
0
: s
2
M
= s
2
R
versus H
1
: s
2
M
,= s
2
R
When H
0
is true F = ^ s
2
M
=^ s
2
R
- F
(NMKM;NRKR)
Transformed model for var(e
i
) = s
2
i
= s
2
x
i
y
i
=

x
i
_
= b
1
1=

x
i
_
( ) b
2
x
i
=

x
i
_
( ) e
i
=

x
i
_
Estimating the variance function
ln(^e
2
i
) = ln(s
2
i
) v
i
= a
1
a
2
z
i2
a
S
z
iS
v
i
Grouped data
var(e
i
) = s
2
i
=
s
2
M
i = 1; 2; . . . ; N
M
s
2
R
i = 1; 2; . . . ; N
R
_
Transformed model for feasible generalized least squares
y
i
_

^ s
i
_
= b
1
1
_

^ s
i
_
_ _
b
2
x
i
_

^ s
i
_
_ _
e
i
_

^ s
i
_
Regression with Stationary Time Series Variables
Finite distributed lag model
y
t
=a b
0
x
t
b
1
x
t1
b
2
x
t2
b
q
x
tq
v
t
Correlogram
r
k
= (y
t
y)(y
tk
y)= (y
t
y)
2
For H
0
: r
k
= 0; z =

T
_
r
k
- N(0; 1)
LM test
y
t
=b
1
b
2
x
t
r^e
t1
^v
t
Test H
0
: r =0 with t-test
^e
t
=g
1
g
2
x
t
r^e
t1
^v
t
Test using LM=T R
2
AR(1) error y
t
=b
1
b
2
x
t
e
t
e
t
= re
t1
v
t
Nonlinear least squares estimation
y
t
= b
1
(1 r) b
2
x
t
ry
t1
b
2
rx
t1
v
t
ARDL(p, q) model
y
t
= d d
0
x
t
d
l
x
t1
d
q
x
tq
u
l
y
t1
u
p
y
tp
v
t
AR(p) forecasting model
y
t
= d u
l
y
t1
u
2
y
t2
u
p
y
tp
v
t
Exponential smoothing ^y
t
= ay
t1
(1 a)^y
t1
Multiplier analysis
d
0
d
1
L d
2
L
2
d
q
L
q
= (1 u
1
L u
2
L
2
u
p
L
p
)
(b
0
b
1
L b
2
L
2
)
Unit Roots and Cointegration
Unit Root Test for Stationarity: Null hypothesis:
H
0
: g = 0
Dickey-Fuller Test 1 (no constant and no trend):
Dy
t
= gy
t1
v
t
Dickey-Fuller Test 2 (with constant but no trend):
Dy
t
= a gy
t1
v
t
Dickey-Fuller Test 3 (with constant and with trend):
Dy
t
= a gy
t1
lt v
t
Augmented Dickey-Fuller Tests:
Dy
t
= a gy
t1

m
s=1
a
s
Dy
ts
v
t
Test for cointegration
D^e
t
= g^e
t1
v
t
Random walk: y
t
= y
t1
v
t
Random walk with drift: y
t
= a y
t1
v
t
Random walk model with drift and time trend:
y
t
= a dt y
t1
v
t
Panel Data
Pooled least squares regression
y
it
= b
1
b
2
x
2it
b
3
x
3it
e
it
Cluster robust standard errors cov(e
it
, e
is
) = c
ts
Fixed effects model
y
it
= b
1i
b
2
x
2it
b
3
x
3it
e
it
b
1i
not random
y
it
y
i
= b
2
(x
2it
x
2i
) b
3
(x
3it
x
3i
) (e
it
e
i
)
Random effects model
y
it
=b
1i
b
2
x
2it
b
3
x
3it
e
it
b
it
=b
1
u
i
random
y
it
ay
i
=b
1
(1a) b
2
(x
2it
ax
2i
) b
3
(x
3it
ax
3i
) v
+
it
a=1s
e
_

Ts
2
u
s
2
e
_
Hausman test
t = (b
FE;k
b
RE;k
)
_

var(b
FE;k
)

var(b
RE;k
)
_ _
1=2