Handout 6 How Well does the Sample Mean Estimate the Population Mean ?

As previously discussed, the point of taking a sample and computing a sample mean is to estimate the population mean. But how well does the sample mean estimate the population mean ? Of course, were hoping for a quantitative answer.... In this handout, we examine this question for the case of a simple random sample. ( Chapter 8 of the text discusses more complicated sampling schemes . ) Definition A simple random sample consists of a fixed number of independent observations of the same random variable. Remarks Saying that one is observing the same random variable means that each observation takes on the same set of values with the same respective probabilities. Think of a random variable that is one if a coin flip comes up heads, and zero if a coin flip comes up tails. A sequence of 20 zeros and ones generated by flipping the same ( not necessarily fair ) coin is a simple random sample of size 20. In applications, it is often far easier to say take a simple random sample than to actually do it ! We will have a MINITAB project that addresses the practical issues of how to actually form a simple random from a pre-existing list. Some preliminary observations : ( 1 ) Stretch your powers of abstraction a bit, and think of a higher-level random experiment that consists in making n independent observations of the outcome of some random experiment, for some fixed sample size n. Then the sample mean is itself a random variable.

( 2 ) Granted that the sample mean is a random variable, it must have its own mean and variance. So, if the random variable being observed is X, then is a random variable with some mean and some standard deviation .

( 3 ) The standard deviation of the sample mean,

, measures the dispersion of

the sample mean. Now think about it a red-hot minute : dispersion is the last thing one wants to have in an estimator having a good estimator means that the estimates are not all spread out ; you want the estimates to fall right around
:, the population mean. So having a good estimator means

should be small.

(4)

Intuition suggests that , everything else being equal, the larger the sample size the better the sample mean estimates the population mean. Any reasonable result should accord with this intuition.

And now for the major result of this section. The Standard Error Theorem Suppose X is a random variable with mean : and standard deviation F , then if is the sample mean corresponding to a simple random sample of size n, then

and Remark The standard deviation of the sample mean,

, is often called the standard

error of estimate or just plain standard error on casual Fridays. The standard error is a quantitative measure of how well the sample mean estimates the population mean. Proof Let { X1, X2, , Xn } be a simple random sample , and write the sample mean as

Then using the properties of expectation previously discussed in Handout 5 :

Now lets derive the variance of the sample mean using the properties of variance previously discussed in Handout 5 :

Hence

, as claimed.

You are not responsible for knowing this proof ; you are responsible for knowing the result - which is quite central to the study of estimation. Math. Lab. ( Like chem. lab. only theres no risk of getting burnt ! ) To illustrate the Standard Error Theorem we consider the following experiment using a pair of wizards dice to generate 100 simple random samples of size n for the random variable W described at the end of Handout 5, where n = 1,4,16 , .... If the Standard Error Theorem is really, really true, then increasing the sample size by a factor of 4 ought to decrease the standard deviation of the sample means by a factor of about ( Why about ? ) Take a look at the table of results, and see if the results confirm this ....

n=

mean of 100 sample means 48.37 49.87 49.87 50.22 49.52 49.60 49.53

std. dev. of 100 sample means 28.32 13.39 7.60 3.54 1.80 0.85 0.46

min. max. ( of 100 sample means ) 0 16.25 32.69 42.92 44.37 47.74 48.26 99 75.75 69.62 59.06 52.63 52.56 50.48

1 4 16 64 256 1024 4096