Two Samples z-test

We know how to deal with one sample z-test.

Now we look at two independent samples.

Introduction
Suppose we want to compare two samples: look at the
difference between their averages.
For example: in box A, the average is 110, and the SD of
the box is 60.
In box B, the average is 90, and the SD is 40.
Now we draw one sample at random independently
from each of the boxes.
From box A, 400 draws are made.
From box B, 100 draws are made.

Introduction
Then in box A, the SE for average is .
In box B, the SE for average is .
Since we want to look at the difference between the
averages, we first can figure out the expected
difference: 110 90 = 20.
But we need the SE for difference, and it is not just the
difference between the SEs for average.
The standard error for the difference of two independent
quantities is , where is the SE for the first quantity and
is the SE for the second.
So in this example, the SE for the difference is .

Introduction
If
the sample average from box A is 120, and the sample
average from box B is 95.
Then we can detect whether the difference between the
averages of the two independent samples is due to chance or
not.
Since it is about the average or sum of the box, we can use the
z-test.
Recall the z-statistic can be computed by .
From the two sample averages, we see that the observed
difference is 120 95 = 25. So , and from the normal table the
P-value is about 16%, not significant.
Therefore, the difference is due to chance.

Example 1
The National Assessment of Education Progress (NAEP)
monitors trends in school performance.
Each year, NAEP administers tests on several subjects to a
nationwide sample of 17-year-olds who are in school. The
reading test was given in 1990 and again in 2004.
The average score went down from 290 to 285. The difference
is 5 points. Is the difference real or due to chance?
In fact, the NAEP design was quite complicated, but we could
consider a simplified version: the test was administered to a
nationwide simple random sample of 1,000 students.
In 1990, the SD of the scores was 40. In 2004, the SD was 37.

Example 1
To
set up a test of significance, we first make the null hypothesis:
The average score in 2004 was equal to the average score in
1990.
Then the alternative hypothesis is: the average score went down.
We use the z-statistic defined by .
Based on the null, we see that the expected difference is 0. The
observed difference is 285 290 = -5. So we need the SE for
difference.
In 2004, the SE for average is . In 1990, the SE for average is
Therefore, the SE for difference is .

Example 1
Finally, . That is, the difference between 2004 and 1990
was about 2.8 SEs below the value expected on the null.
The P-value is about 0.25% (< 1%), which is highly
significant.
So we reject the null, and conclude that the average
score did go down.
Note: More detailed analysis of the data suggests that
each extra year of schooling is associated with about a
6-point increase test scores. A 5-point decline is
worrisome.

Example 2
In 1999, NAEP found that 13% of the 17-year-old
students had taken calculus, compared to 17% in 2004.
Suppose they took a simple random sample of size
1,000 in each time.
Is the difference real, or due to chance?

Example 2
To
set up a test of significance, we then make the null
hypothesis:
The percentage in 2004 was equal to the percentage in 1999.
The alternative is: the percentage in 2004 increased.
Again, we use the z-statistic by .
Based on the null, the expected difference is 0%.
The observed difference is 17% - 13% = 4%.
We compare the percentages of two 0-1 boxes, the contents
of the boxes can be estimated: in 2004, the SE for
percentage is ; in 1999, similarly, the SE for percentage is
about 1.1%.

Example 2
So the SE for difference is .
Finally, the z statistic is about .
Therefore, from the normal table, the P-value is about
0.6%, which is highly significant.
Again, we reject the null and conclude that the
percentage in 2004 increased.

Example 3
The model can also be applied to (randomized control)
experiments:
There are 200 subjects in a small clinical trial on vitamin C.
Half the subjects are assigned at random to treatment (2,000
mg of vitamin C daily) and half to control (2,000 mg of
placebo).
Over the period of the experiment, the treatment group
averaged 2.3 colds, and the SD was 3.1.
The controls did a little worse: they averaged 2.6 colds and the
SD was 2.9.
Q: Is the difference in averages statistically significant?

Example 3
To set up a test of significance, we again start with the
null hypothesis:
There is no difference between the treatment group and
control group. The average colds are equal to each
other.
Then the alternative hypothesis is: the vitamin C works,
and the treatment group really does better. The average
colds in the treatment group is essentially lower than
the control group.
Again, we use the z-test, .
Based on the null, the expected difference is 0.

Example 3
The
observed difference is 2.3 2.6 = -0.3.

Now just pretend that we have two independent samples

drawn at random with replacement. The SE for the
treatment average is . The SE for the control average is .
Then the SE for the difference is about .
Finally, the z statistic is about .
Therefore, from the normal table, the P-value is about 24%,
which is far away from 5%.
Then we can conclude that the difference could be easily
due to chance, and vitamin C has no effect. (A few too many
susceptible people were assigned to the control group.)

Remark
In example 3, there are 2 problems in dealing with the
SE:
1. The draws are made without replacement. (Inflates
the SE.)
2. The two averages are dependent. (Cuts the SE back
down.)
But the computation can still be applied, even though
there are problems. This is because these two problems
cancel out each other.
Therefore, the two-sample z-test here still applies.