Monte Carlo Simulation: SLR

Exercise 1
)

( E
3
Estimating the Error Variance
We dont know what the error variance,
2
,
is, because we dont observe the errors, u
i


What we observe are the residuals,
i

We can use the residuals to form an estimate
of the error variance
4
Error Variance Estimate (cont)



2 /
2
1

is of estimator unbiased an Then,

and

for similarly and

of value expected The

2 2
2
1 0 0
1 1 0 0
1 0 1 0
1 0

n SSR u
n
u equals
x u
x u x
x y u
i
i
i i
i i i
i i i

5
Exercise 2
The OLS estimates of
1
and
0
are unbiased if





Proof of unbiasedness depends on our 4 assumptions
if any assumption fails, then OLS is not necessarily
unbiased

Remember unbiasedness is a description of the
estimator in a given sample we may be near or
far from the true parameter

1 1
0 0
)

(
)

E
E
6
Exercise 2: Contd:


2
1
2
1
1
2
/

se
,

of error standard the

have then we for

substitute we if

sd that recall
regression the of error Standard

x x
s
i
x

Check if estimated Variance of Beta_hat is same as
true variance of beta_hat
Exercise 4
Exercise 4: Show that the variance of u and
y are same for a given value of x.

The mean of u is zero and the mean of y is
b0+ b1x.

Also show that the variance of u
unconditional on x is same as variance of u
conditional on x.

8
Variance of OLS (cont)
Var(u|x) = E(u
2
|x)- [E(u|x)]
2


E(u|x) = 0, so
2
= E(u
2
|x) = E(u
2
) = Var(u)

Thus
2
is also the unconditional variance, called the error variance

, the square root of the error variance is called the standard
deviation of the error

Can say: E(y|x)=
0
+
1
x and Var(y|x) =
2

The random variables y and e have the same variance because they
differ only by a constant.

Simulate one sample Data
/* simulate data */
data normal;
call streaminit(1234567);
do n = 1 to 40;
if n < 21 then x = 10;
else x = 20;
e = rand('normal',0,50);
y = 100 + 10*x + e;
output;
end;
run;
Print the Dataset
proc print data=normal(firstobs=15 obs=25);
var x y;
title 'Observations from normal DGP';
run;
Compute True Variance of B1_hat

2
2
1

x
s
Var


= 2500/1000
=2.5
Computing True Variance of b1_hat
proc means data=normal css;
var x;
title 'corrected ss for x or (x-xbar)^2';
run;
/* true parameter values */
data truevar;
sig2 = 2500;
ssx = 1000;
truevar = sig2/ssx;
truese = sqrt(truevar);
run;
Print
proc print data=truevar;
var truevar truese;
title 'true var(b1) and se(b1)';
run;
Regress Y on X
proc reg data=normal outest=est tableout mse;
model y = x;
title 'regression with normal data';
title2 'true intercept = 100 & slope = 10';
run;
Check the Results
here you should find that estimated intercept
and slope are 112 and 8.9 approx.
respectively.

esimtated variance of the error term is 2803
against true value of 2500;

true standard error of b1 is 1.58 and the
estimated standard error is 1.67;
/* print est data set */


proc print data=est;
title 'OUTEST data';
Monte Carlo Experiment:
simulate data 10,000 times
data normal;
call streaminit(1234567);
do sample = 1 to 10000;
do n = 1 to 40;
if n < 21 then x = 10;
else x = 20;
e = rand('normal',0,50);
y = 100 + 10*x + e;
output;
end;
End;
run;

/* regression with NOPRINT option */

proc reg noprint data=normal outest=est tableout MSE;
model y = x;
by sample;
run;
data parm;
set est;
if _type_='PARMS';
b0 = intercept;
b1 = x;
sig2 = _MSE_;
keep sample b0 b1 sig2;
run;
Monte Carlo summary statistics
proc means data=parm mean var std min max p1 p99;
var b0 b1 sig2;
title 'Monte Carlo summary statistics';
run;
Results summary
sample variance of b1 is 2.56 when compared
to true variance of 2.5,

which illustrates what variance of b1 actually
measures- the sampling variation of the least
squares estimates in repeated(many) samples.

the average estimate of error variance is 2493
compared to the true value of 2500.;
summarize slope estimates & plot
histogram
proc univariate data = parm;
var b1;
histogram/normal;
title 'Sampling distribution of b1';
title2 'Errors Normal';
run;
Check if variance estimated is
unbiased
data se;
set est;
if _type_='STDERR';
se = x;
varb1 = se**2;
keep sample varb1 se;
run;

proc means data=se mean;
var varb1 se;
title 'average value estimated var(b1): normal data';
title2 'true var(b1)= 2.50';
run;
Finally check the estimator of the
variance of b1_hat
That is standard error of b1_hat
Where do you find this??

Exercise 4
proc sort data=normal; by x; run;

proc means data=normal; by x;
var y e;
run;

*check here : the standard deviation of e is the same
irrespective of x but not y;

proc means data=normal;
var y e; run;