Section 1 Notes GSI: Kyle Emerick

EEP/IAS 118 September 1st, 2011

Derivation of OLS Estimator

In class we set up the minimization problem that is the starting point for deriving the formulas for the OLS intercept and slope coecient. That problem was,
N 0 ,1 i=1

min

(yi 0 1 xi )2 .

(1)

As we learned in calculus, a univariate optimization involves taking the derivative and setting equal to 0. Similarly, this minimization problem above is solved by setting the partial derivatives equal to 0. That is, take the derivative of (1) with respect to 0 and set it equal to 0. We then do the same thing for 1 . This gives us, N W = 2(yi 0 1 xi ) = 0 (2) 0
i=1

and, W = 1
N

2xi (yi 0 1 xi ) = 0
i=1

(3)

N Note that I have used W to denote i=1 (yi 0 1 xi )2 . Now our task is to solve (2) and (3) using some algebra tricks and some properties of summations. Lets start with the rst order condition for 0 (this is N 0 1 xi = 0. Now lets rearrange Equation (2)). We can immediately get rid of the 2 and write i=1 yi N this expression and make use of the algebraic fact that i=1 yi = N y . This leaves us with,

N 0 = N y N 1 x.

(4)

We simply divide everything by N and amazing, we have the formula that Professor Sadoulet gave in lecture! That is, 0 = y 1 x. (5) 1 . This one is a bit more tricky. We can rst get rid of the 2 and rearrange Now lets consider solving for N Equation (3) to get i=1 xi yi 0 xi 1 x2 = 0. Now lets substitute our result for 0 into this expression i and this gives us,
N

xi yi ( 1 x)xi 1 x2 = 0 y i
i=1

(6)

Note that the summation is applying to everything in the above equation. We can distribute the sum to each term to get,
N N N N

xi yi y
i=1 i=1

x i + 1 x
i=1

x i 1
i=1

x2 = 0. i

(7)

We have of course used the property that you can always pull a constant term out in front of a summation. N N Lets again use the property that i=1 yi = N y (and of course this also means that i=1 xi = N x). We apply these facts to Equation (7) and solve for 1 . This gives, 1 =
N i=1 xi yi N xy . N 2 N x2 i=1 xi

(8)

Doesnt quite look like the formula from class, right? Well, let us just use a couple more tricks. You can N N either look up or derive for yourself that i=1 (xi x)(yi y ) = i=1 xi yi N xy . You can also easily N N 2 2 2 derive that i=1 (xi x) = i=1 xi N x . These two can be derived very easily using algebra. Now we substitute these two properties into (8) and we have something that looks very, very familiar: 1 = All done!
Page 1
N i=1 (xi x)(yi N 2 i=1 (xi x)

y)

(9)