Simple Linear Regression

Definition OLS Estimates Goodness of fit Units of measurement and functional form Expected values and variances of the OLS model

What is Simple Linear Regression?

X explains Y : Y=0 +1X +u u is the error term, or disturbance, or unobservable
Y Dependent variable Explained variable Response variable Predicted variable Regressand X Independent variable Explanatory variable Control variable Predictor variable Regressor
2

X has a linear effect on Y

Y=0 +1X+u Y= 1 X but only if u=0, i.e. all other factors are fixed
Example 1: yield= 0 +1fertiliser +u yield= 1 fertiliser 1measures the effect of a 1 unit change in fertiliser on yield, holding all other factors fixed , i.e. iff u=0 Example 2: wage= 0 +1educ+u wage= 1 educ 1measures the effect of a 1 unit change in years of education on hourly wage, holding all other factors fixed , i.e. iff u=0
3

Two assumptions about u

Easy: E(u)=0 : average value of u in the population is zero
can normalise all unobservables so that mean is zero (e.g. land quality, ability)

Harder: E(u|x)=E(u)=0: average value of u does not depend on value of X E(xu)=0

does E(ability|8)=E(ability|16)=E(ability)=0? Probably not: average ability among people with more education is probably higher; so need to include a measure of ability in the model NB if true then cov(x,u)=0, i.e. E(xu)=0

The OLS estimates

Y=0 +1x +u rearranging gives u=y- 0 -1x If E(u)=0 and E(xu)=0, then E(y- 0 -1x)=0 and E[x(y- 0 -1x)]=0 Choose estimates of 0 and 1 to solve these 2 equations

( x x )( y y ) = (x x)
i i 2 i

0 = y 1 x

proof not required

5

Fitted Values and Residuals

yi = 0 + 1 xi

yi

u i = y i yi

yi
6

Example 1: are CEO annual salaries related to firm performance?

y=annual salary in $000s X= average return on equity (net income as a % of equity) 1 measures change in annual salary in 000s when return on equity increases by 1 percentage point: 1>0?

salary = 0 + 1roe + u

s = 963 + 18.5701roe

If roe=0 then predicted salary is $963,000 If roe rises by 1 percentage point, roe=1 so salary=18.501, i.e. salary changes by $18,501
7

Example 2: do wages reflect education?

y=hourly wage $s X= years of schooling

wage = 0 + 1educ + u

1 measures change in hourly wage in $s when years of education increases by 1 year: 1>0?

w = 0.90 + 0.54educ
If educ=0 then predicted hourly wage is -$0.90, this has note an economical meaning If educ rises by 1 year, educ=1 so wage=0.54, i.e. wage rises by $0.54 an hour if educ rises by 4 years, educ=4 so wage=4*0.54, i.e. wage rises by $2.16 an hour
8

Example 3: is it worth spending on electoral campaigns?

y=percentage of vote received by candidate A X= percentage of total campaign expenditure on candidate A 1 measures change in percentage share of vote for A in % points when campaign spending share of A increases by 1 % point: 1>0?

voteA = 0 + 1shareA + u

v = 26.81 + 0.464 shareA

If shareA=0 then predicted share of the vote for A is 26.81% If shareA rises by 1 percentage point, shareA=1 so voteA=0.464, i.e. share of vote for A rises by 0.464 % point
9

Goodness of fit
SST = [ yi y ]2
i =1
n

total sample variation Sum Squared Total variation of fitted values Sum Squared Explained
n

SSE = [ yi y ]2
i =1

SSR = [ yi yi ]2 = ui2
i =1 i =1

variation of residuals Sum Squared of Residual

10

R2 : coefficient of determination
SST = SSE + SSR SSE SSR 1= + SST SST SSE SSR 2 = 1 R = SST SST

R2=0.0132: X explains only 1.3% of variation in Y R2=0.856: X explains 85.6% of the variation in Y 11

Changing units
E.g. change units from thousands of $ to hundreds of dollars, or from $ to OLS estimates change accordingly:
if units of dep var are multiplied or divided by a constant c, then so are 0 and 1 if units of indep var are multiplied or divided by a constant c, then so are 0 and 1
12

Changing functional form

Nonlinearities: semi-log
log(wage)= 0 + 1educ+u %wage1001educ

log y = 1x

100 log y = 100 1x %y 100 1x

percentage increase in wage for an increase of 1 year education

(log wage ) = 0.584 + 0.083educ

wage increases by 8.3 percent for every additional year of education Semi-log model allows for increasing returns to education
13

Changing functional form

Nonlinearities: double-log or constant elasticity model
log(salary)= 0 + 1log(sales)+u %salary= 1 % sales percentage increase in salary for an increase of 1% in sales

log( salary ) = 4.822 + 0.257 log( sales )

salary increases by 0.257 percent if sales rise by 1% Double-log model allows for a constant elasticty
14

Interpretation of coefficients
Model
levellevel levellog loglevel log-log

Dep Var
y y log(y) log(y)

Indep Var
x log(x) x log(x)

Interpretation of 1
y=1x y=(1/100)%x %y=(1001)x %y=1%x

15

Expected values and variances of OLS estimators

Unbiased
expected value of the distribution of OLS estimates equals the true population parameter E ( ) = and E ( ) =
0 0 1 1

bias can arise if functional form is not linear in its parameters, if sample is not random, or if there is no sample variation. Most importantly, if E(u|X) is not 0, e.g. omitted variables correlated with X
16