Beruflich Dokumente
Kultur Dokumente
• We use sixteen identical slips of paper, four marked with A, four with
B, and so on to D. We put the slips of paper into a basket and mix them
thoroughly. For each unit, we draw a slip of paper from the basket and
use the treatment marked on the slip.
• Treatment A is assigned to the first four units we happen to encounter,
treatment B to the next four units, and so on.
• As each unit is encountered, we assign treatments A, B, C, and D based
on whether the “seconds” reading on the clock is between 1 and 15, 16
and 30, 31 and 45, or 46 and 60.
• Toss a coin for every patient; heads—the patient gets the drug, tails—
the patient gets surgery.
• Make up a basket with 50 red balls and 50 white balls well mixed
together. Each patient gets a randomly drawn ball; red balls lead to
surgery, white balls lead to drug therapy.
Note that for coin tossing the numbers of patients in the two treatment groups
are random, while the numbers are fixed for the colored ball scheme.
Here is how randomization has helped us. No matter which features of
the population of experimental units are associated with our response, our
randomizations put approximately half the patients with these features in
each treatment group. Approximately half the men get the drug; approxi- Randomization
mately half the older patients get the drug; approximately half the stronger balances the
patients get the drug; and so on. These are not exactly 50/50 splits, but the population on
deviation from an even split follows rules of probability that we can use when average
making inference about the treatments.
This example is, of course, an oversimplification. A real experimental
design would include considerations for age, gender, health status, and so
on. The beauty of randomization is that it helps prevent confounding, even
for factors that we do not know are important.
16 Randomization and Design
• If the experimental units are not used simultaneously, you can random-
ize the order in which they are used.
• If the experimental units are not used at the same location, you can
randomize the locations at which they are used.
• If you use more than one measuring instrument for determining re-
sponse, you can randomize which units are measured on which instru-
ments.
When we anticipate that one of these might cause a change in the response,
we can often design that into the experiment (for example, by using blocking;
see Chapter 13). Thus I try to design for the known problems, and randomize
everything else.
and choosing random subsets. When the sample sizes for the subsets are fixed
and known (as they usually are), we will be able to choose random subsets
by first choosing random orders.
Randomization methods can be either physical or numerical. Physical
randomization is achieved via an actual physical act that is believed to pro-
duce random results with known properties. Examples of physical random-
Physical ization are coin tosses, card draws from shuffled decks, rolls of a die, and
randomization tickets in a hat. I say “believed to produce random results with known prop-
erties” because cards can be poorly shuffled, tickets in the hat can be poorly
mixed, and skilled magicians can toss coins that come up heads every time.
Large scale embarrassments due to faulty physical randomization include
poor mixing of Selective Service draft induction numbers during World War
II (see Mosteller, Rourke, and Thomas 1970). It is important to make sure
that any physical randomization that you use is done well.
Physical generation of random orders is most easily done with cards or
tickets in a hat. We must order N objects. We take N cards or tickets,
Physical random numbered 1 through N , and mix them well. The first object is then given the
order number of the first card or ticket drawn, and so on. The objects are then sorted
so that their assigned numbers are in increasing order. With good mixing, all
orders of the objects are equally likely.
Once we have a random order, random subsets are easy. Suppose that
the N objects are to be broken into g subsets with sizes n1 , . . ., ng , with
Physical random n1 + · · · + ng = N . For example, eight students are to be grouped into one
subsets from group of four and two groups of two. First arrange the objects in random
random orders order. Once the objects are in random order, assign the first n1 objects to
group one, the next n2 objects to group two, and so on. If our eight students
were randomly ordered 3, 1, 6, 8, 5, 7, 2, 4, then our three groups would be
(3, 1, 6, 8), (5, 7), and (2, 4).
Numerical randomization uses numbers taken from a table of “random”
Numerical numbers or generated by a “random” number generator in computer software.
randomization For example, Appendix Table D.1 contains random digits. We use the table
or a generator to produce a random ordering for our N objects, and then
proceed as for physical randomization if we need random subsets.
We get the random order by obtaining a random number for each object,
and then sorting the objects so that the random numbers are in increasing
order. Start arbitrarily in the table and read numbers of the required size
sequentially from the table. If any number is a repeat of an earlier number,
replace the repeat by the next number in the list so that you get N different
Numerical numbers. For example, suppose that we need 5 numbers and that the random
random order numbers in the table are (4, 3, 7, 4, 6, 7, 2, 1, 9, . . .). Then our 5 selected
numbers would be (4, 3, 7, 6, 2), the duplicates of 4 and 7 being discarded.
2.4 Randomization for Inference 19
Now arrange the objects so that their selected numbers are in ascending order.
For the sample numbers, the objects, A through E would be reordered E, B,
A, D, C. Obviously, you need numbers with more digits as N gets larger.
Getting rid of duplicates makes this procedure a little tedious. You will
have fewer duplicates if you use numbers with more digits than are abso-
lutely necessary. For example, for 9 objects, we could use two- or three-digit Longer random
numbers, and for 30 objects we could use three- or four-digit numbers. The numbers have
probabilities of 9 random one-, two-, and three-digit numbers having no du- fewer duplicates
plicates are .004, .690, and .965; the probabilities of 30 random two-, three-,
and four-digit numbers having no duplicates are .008, .644, and .957 respec-
tively.
Many computer software packages (and even calculators) can produce
“random” numbers. Some produce random integers, others numbers be-
tween 0 and 1. In either case, you use these numbers as you would numbers
formed by a sequence of digits from a random number table. Suppose that
we needed to put 6 units into random order, and that our random number
generator produced the following numbers: .52983, .37225, .99139, .48011,
.69382, .61181. Associate the 6 units with these random numbers. The sec-
ond unit has the smallest random number, so the second unit is first in the
ordering; the fourth unit has the next smallest random number, so it is second
in the ordering; and so on. Thus the random order of the units is B, D, A, F,
E, C.
The word random is quoted above because these numbers are not truly
random. The numbers in the table are the same every time you read it; they
don’t change unpredictably when you open the book. The numbers produced
by the software package are from an algorithm; if you know the algorithm
you can predict the numbers perfectly. They are technically pseudorandom
numbers; that is, numbers that possess many of the attributes of random num- Pseudorandom
bers so that they appear to be random and can usually be used in place of numbers
random numbers.
Bezjak and Knez (1995) provide data on the length of time it takes garment
workers to runstitch a collar on a man’s shirt, using a standard workplace and
a more ergonomic workplace. Table 2.1 gives the “auxiliary manual time”
per collar in seconds for 30 workers using both systems.
One question of interest is whether the times are the same on average
for the two workplaces. Formally, we test the null hypothesis that the aver-
age runstitching time for the standard workplace is the same as the average
runstitching time for the ergonomic workplace.
2.4 Randomization for Inference 21
A paired t-test is the standard procedure for testing this null hypothesis.
We use a paired t-test because each worker was measured twice, once for Paired t-test for
each workplace, so the observations on the two workplaces are dependent. paired data
Fast workers are probably fast for both workplaces, and slow workers are
slow for both. Thus what we do is compute the difference (standard − er-
gonomic) for each worker, and test the null hypothesis that the average of
these differences is zero using a one sample t-test on the differences.
Table 2.2 gives the differences between standard and ergonomic times.
Recall the setup for a one sample t-test. Let d1 , d2 , . . ., dn be the n differ-
ences in the sample. We assume that these differences are independent sam-
ples from a normal distribution with mean µ and variance σ 2 , both unknown.
Our null hypothesis is that the mean µ equals prespecified value µ0 = 0
(H0 : µ = µ0 = 0), and our alternative is H1 : µ > 0 because we expect the
workers to be faster in the ergonomic workplace.
The formula for a one sample t-test is
d¯ − µ0
t= √ ,
s/ n
where d¯ is the mean of the data (here the differences d1 , d2 , . . ., dn ), n is the The paired t-test
sample size, and s is the sample standard deviation (of the differences)
v
u n
u 1 X
s=t (di − d¯)2 .
n − 1 i=1
If our null hypothesis is correct and our assumptions are true, then the t-
statistic follows a t-distribution with n − 1 degrees of freedom.
The p-value for a test is the probability, assuming that the null hypothesis
is true, of observing a test statistic as extreme or more extreme than the one The p-value
we did observe. “Extreme” means away from the the null hypothesis towards
the alternative hypothesis. Our alternative here is that the true average is
larger than the null hypothesis value, so larger values of the test statistic are
extreme. Thus the p-value is the area under the t-curve with n − 1 degrees of
freedom from the observed t-value to the right. (If the alternative had been
µ < µ0 , then the p-value is the area under the curve to the left of our test
22 Randomization and Design
statistic. For a two sided alternative, the p-value is the area under the curve
at a distance from 0 as great or greater than our test statistic.)
To illustrate the t-test, let’s use the data for the last 10 workers and all
30 workers. Table 2.3 shows the results. Looking at the last ten workers,
the p-value is .46, meaning that we would observe a t-statistic this larger or
larger in 46% of all tests when the null hypothesis is true. Thus there is little
evidence against the null here. When all 30 workers are considered, the p-
value is .074; this is mild evidence against the null hypothesis. The fact that
these two differ probably indicates that the workers are not listed in random
order. In fact, Figure 2.1 shows box-plots for the differences by groups of ten
workers; the lower numbered differences tend to be greater.
Now consider a randomization-based analysis. The randomization null
hypothesis is that the two workplaces are completely equivalent and merely
Randomization act to label the responses that we observed. For example, the first worker
null hypothesis had responses of 4.90 and 3.87, which we have labeled as standard and er-
gonomic. Under the randomization null, the responses would be 4.90 and
3.87 no matter how the random assignment of treatments turned out. The
only thing that could change is which of the two is labeled as standard, and
which as ergonomic. Thus, under the randomization null hypothesis, we
could, with equal probability, have observed 3.87 for standard and 4.90 for
ergonomic.
What does this mean in terms of the differences? We observed a differ-
Differences have ence of 1.03 for worker 1. Under the randomization null, we could just as
random signs easily have observed the difference -1.03, and similarly for all the other dif-
under ferences. Thus in the randomization analogue to a paired t-test, the absolute
randomization values of the differences are taken to be fixed, and the signs of the differ-
null ences are random, with each sign independent of the others and having equal
probability of positive and negative.
To construct a randomization test, we choose a descriptive statistic for
the data and then get the distribution of that statistic under the randomization
null hypothesis. The randomization p-value is the probability (under this
randomization distribution) of getting a descriptive statistic as extreme or
more extreme than the one we observed.
2.4 Randomization for Inference 23
1.5
T
i
m 1
e
d
i 0.5
f
f
e 0
r
e
n
c -0.5
e
-1
1 2 3
Group of 10
For this problem, we take the sum of the differences as our descriptive
statistic. (The average would lead to exactly the same p-values, and we could
also form tests using the median or other measures of center.) Start with Randomization
the last 10 workers. The sum of the last 10 observed differences is .23. To statistic and
get the randomization distribution, we have to get the sum for all possible distribution
combinations of signs for the differences. There are two possibilities for
each difference, and 10 differences, so there are 210 = 1024 different equally
likely values for the sum in the randomization distribution. We must look at
all of them to get the randomization p-value.
0.16
0.14
0.12
D
e 0.1
n
s
i 0.08
t
y 0.06
0.04
0.02
0
-6 -4 -2 0 2 4 6
Sum of differences
.62 1.75 .71 .21 .01 .42 -.21 .42 .43 .82
Only one of these values is negative (-.21), and seven of the positive differ-
ences have absolute value greater than .21. Any change of these seven values
can only make the sum less, so we don’t have to consider changing their
signs, only the signs of .21, .01, and -.21. This is a much smaller problem,
and it is fairly easy to work out that four of the 8 possible sign arrangements
for testing three differences lead to sums as large or larger than the observed
sum. Thus the randomization p-value is 4/1024 = .004, similar to the .007
p-value we would get if we used the t-test.
Looking at the entire data set, we have 230 = 1, 073, 741, 824 different
Subsample the sets of signs. That is too many to do comfortably, even on a computer. What
randomization is done instead is to have the computer choose a random sample from this
distribution complete distribution by choosing random sets of signs, and then use this
sample for computing randomization p-values as if it were the complete dis-
tribution. For a reasonably large sample, say 10,000, the approximation is
usually good enough. I took a random sample of size 10,000 and got a p-
value .069, reasonably close to the t-test p-value. Two additional samples
of 10,000 gave p-values of .073 and .068; the binomial distribution suggests
2.4 Randomization for Inference 25
where y 1• and y2• are the means of the first and second samples, n1 and n2
are the sample sizes, and s2p is the pooled estimate of variance defined by
sP P
n1
i=1 (y1i − y1• )2 + ni=1
2
(y2i − y 2• )2
sp = .
n1 + n2 − 2
26 Randomization and Design
If our null hypothesis is correct and our assumptions are true, then the t-
statistic follows a t-distribution with n1 + n2 − 2 degrees of freedom. The
p-value for our one-sided alternative is the area under the t-distribution curve
with n1 + n2 − 2 degrees of freedom that is to the right of our observed
t-statistic.
For these data y 1• = 4.775, y 2• = 5.825, sp = .446, and n1 = n2 = 4.
The t-statistic is then
5.825 − 4.775
t= p = 3.33,
.446 1/4 + 1/4
and the p-value is .008, the area under a t-curve with 6 degrees of freedom to
the right of 3.33. This is strong evidence against the null hypothesis, and we
would probably conclude that the null is false.
Now consider a randomization analysis. The randomization null hypoth-
Randomization esis is that growing time treatments are completely equivalent and serve only
null hypothesis as labels. In particular, the responses we observed for the 8 units would be
the same no matter which treatments had been applied, and any subset of four
units is equally likely to be the 15-day treatment group. For example, under
the randomization null wth the 15-day treatment, the responses (4.3, 4.6, 4.8,
5.4), (4.3, 4.6, 5.3, 5.7), and (5.4, 5.7, 6.0, 6.3) are all equally likely.
To construct a randomization test, we choose a descriptive statistic for
the data and then get the distribution of that statistic under the randomization
null hypothesis. The randomization p-value is the probability (under this
randomization distribution) of getting a descriptive statistic as extreme or
more extreme than the one we observed.
For this problem, we take the average response at 28 days minus the aver-
Randomization age response at 15 days as our statistic. The observed value of this statistic is
statistic and 1.05. There are 8 C4 = 70 different ways that the 8 plants can be split between
distribution the two treatments. Only two of those 70 ways give a difference of averages
as large as or larger than the one we observed. Thus the randomization p-
Randomization value is 2/70 = .029. This p-value is a bit bigger than that computed from
p-value the t-test, but both give evidence against the null hypothesis. Note that the
smallest possible randomization p-value for this experiment is 1/70 = .014.
large and the assumptions of the standard methods are met. For small sample
sizes, randomization inference is coarser, in the sense that there are relatively
few obtainable p-values.
2.6 Problems
Exercise 2.1 We wish to evaluate a new textbook for a statistics class. There are seven
sections; four are chosen at random to receive the new book, three receive the
old book. At the end of the semester, student evaluations show the following
percentages of students rate the textbook as “very good” or “excellent”:
Section 1 2 3 4 5 6 7
Book N O O N N O N
Rating 46 37 47 45 32 62 56
Find the one-sided randomization p-value for testing the null hypothesis that
the two books are equivalent versus the alternative that the new book is better
(receives higher scores).
Exercise 2.2 Dairy cows are bred by selected bulls, but not all cows become pregnant
at the first service. A drug is proposed that is hoped to increase the bulls
fertility. Each of seven bulls will be bred to 2 herds of 100 cows each (a
total of 14 herds). For one herd (selected randomly) the bulls will be given
the drug, while no drug will be given for the second herd. Assume the drug
has no residual effect. The response we observe for each bull is the number
2.6 Problems 29
Reduced pH Neutral pH
256 159 149 54 123 248
Using randomization methods, test the null hypothesis that the two treatments
have equal average numbers of Copepoda versus a two-sided alternative.
Chu (1970) studied the effect of the insecticide chlordane on the ner- Exercise 2.5
vous systems of American cockroaches. The coxal muscles from one meso-
and one metathoracic leg on opposite sides were surgically extracted from
each of six roaches. The roaches were then treated with 50 micrograms of
α-chlordane, and coxal muscles from the two remaining meso- and metatho-
racic legs were removed about two hours after treatment. The Na+ -K+ ATPase
activity was measured in each muscle, and the percentage changes for the six
roaches are given here:
15.3 -31.8 -35.6 -14.5 3.1 -24.5
Test the null hypothesis that the chlordane treatment has not affected the
30 Randomization and Design
Na+ -K+ ATPas activity. What experimental technique (not mentioned in the
description above) must have been used to justify a randomization test?
Problem 2.1 McElhoe and Conner (1986) use an instrument called a “Visiplume” to
measure ultraviolet light. By comparing absorption in clear air and absorp-
tion in polluted air, the concentration of SO2 in the polluted air can be es-
timated. The EPA has a standard method for measuring SO2 , and we wish
to compare the two methods across a range of air samples. The recorded
response is the ratio of the Visiplume reading to the EPA standard reading.
There were six observations on coal plant number 2: .950, .978, .762, .733,
.823, and 1.011. If we make the null hypothesis be that the Visiplume and
standard measurements are equivalent (and the Visiplume and standard labels
are just labels and nothing more), then the ratios could (with equal probabil-
ity) have been observed as their reciprocals. That is, the ratio of .950 could
with equal probability have been 1/.950 = 1.053, since the labels are equiva-
lent and assigned at random. Suppose we take as our summary of the data the
sum of the ratios. We observe .95 + ... + 1.011 = 5.257. Test (using random-
ization methods) the null hypothesis of equivalent measurement procedures
against the alternative that Visiplume reads higher than the standard. Report
a p-value.
Problem 2.2 In this problem, a data set of size 5 consists of the numbers 1 through 5;
a data set of size 6 consists of the numbers 1 through 6; and so on.
(a) For data sets of size 5 and 6, compute the complete randomization distri-
bution for the mean of samples of size 3. (There will be 10 and 20 members
respectively in the two distributions.) How normal do these distributions
look?
(b) For data sets of size 4 and 5, compute the complete randomization distri-
bution for the mean of samples of any size (size 1, size 2, . . ., up to all the
data in the sample). Again, compare these to normal.
(c) Compare the size 5 distributions from parts a) and b). How do they com-
pare for mean, median, variance, and so on.
Question 2.1 Let X1 , X2 , . . ., XN be independent, uniformly distributed, random k-
digit integers (that is, less than 10k ). Find the probability of having no dupli-
cates in N draws.