" You know how dumb the average guy is?

Well, by definition, half of them are even dumber

than that." -- J. R. "Bob" Dobbs

Chapter 13: Experiments and Observational Studies (Pages 293 - 318)

OVERVIEW: We consider observational studies and experiments, emphasizing the importance of
controlled randomized experiments in establishing cause-and-effect relationships. We
discuss the principles of control, randomization, replication, and blocking, and introduce
informally the notion of statistical significance. We look at completely randomized
one- and two-factor designs, as well as blocked and matched designs. We discuss the
importance of control treatments, the use of blinding and placebos, and the problem of
confounding variables.
Observational study: researchers do not assign treatments.

Experiment: imposes treatment on randomly assigned individuals in order to be observed.

Experimental units:
Subjects:
Response variable: needs to be identified and needed to measure in any experiment.
Factor: explanatory variable needed to manipulate in any experiment.
Level:
Treatment: The combination of specific levels from all the factors that an experimental unit receives.
Three principles of experimental design
1.

: used to make sure that sources of variation other than the factors that
we are testing have little to no impact on our response variables.
-We control a factor by assigning subjects to different factor levels b/c we
want to see how the response will change at those different levels.
- We control other sources of variation to prevent them from
changing and
affecting the response variable.

2.

: assigning subjects to treatment randomly in order to use the

methods of statistics to draw conclusions from your study.
- Creates groups that are similar which allows us to equalize the
effects of unknown or uncontrollable sources of variation.

3.
And sometimes:

: repeat experiment on many subjects to reduce chance variation in the results.

4.

: used to minimize variation. A block is a group of experimental units or subjects

that are similar in ways that are expected to affect the response of the treatments.
Treatment is assigned randomly within similar blocks. Blocking are a form of control.

EX. The makers of Frumpies, the breakfast of rug rats, want to improve their marketing, so they
consult you:
a) They first want to know what fraction of children, ages 10 to 13, like their celery flavored cereal.
What kind of study should they perform?

b) They are thinking of introducing a new flavor, maple marshmallow Frumpies, and want to know
whether children will prefer the new flavor to the old one. Design a completely randomized
experiment to investigate this question.

c) They suspect that children who regularly watch the Saturday morning cartoon show starring
Frump, the flying teenage warrior rabbit who eats Frumpies in every episode, may respond
differently to the new flavor. How would you take that into account in your design?
ANS.

We are looking for a difference between the groups. We expect there will be some difference just
because of random variation. The great unresolved issue is how big that difference has to be before we
can attribute it to the effect of the treatment. This is the core issue of

Control group: receive dummy treatment or levels of treatment set at zero.

Placebo: A dummy treatment that can have no physical effect.
Blinding: Any individual associated with an experiment who is not aware of how subjects have been
allocated to treatment groups.
There are two main classes of individuals who can affect the outcome of an experiment:
1. those who
2. those who
Single blind experiment: when all the individuals in either one of these classes is blinded
Double blind experiment: Neither the subjects nor the people who have contact with them know which
treatment a subject received.
The best experiments are usually:
Matched pairs: A common form of blocking for comparing two treatments. The two pairs are "alike."
Common forms of matched pairs include...
-Using random process. One of pair receives treatment, the other a placebo. Pairs
observed at a later time to see if treatment had effect.
-Test scores from a before-after situation. An individual takes a before-test, then gets
some type of treatment, than takes an after-test (similar to the before-test). Purpose is to
see if treatment improves test performance.

Always be alert to the possibility of Lurking variables and Confounding

Lurking variables are a common problem in observational studies, when an apparent association
between two variables is really just common response to a third unseen variable.
A commonly cited example involves a positive association between ice cream sales and drownings.
What is explanatory and what is response?

In fact, the explanatory variable is probably

Another favorite example is the strong positive association between the number of firefighters at a fire
and the amount of damage. Perhaps you shouldn't call the fire department
The lurking variable is

 Confounding is a different issue. Confounding arises when the response we see in an experiment is at
least partially attributable to uncontrolled variables. Confounded variables vary together so that one
cannot tease apart which is responsible for any observed effect. Confounding can occur due to poor
design in an experiment.

Ethics
Take a look at this conclusion, from Newsweek magazine.
Of all pre-college curricula, the highest level of mathematics one studies in secondary school has the
strongest continuing influence on bachelor's degree completion. Finishing a course beyond the level of
Algebra 2 (for example, trigonometry or pre-calculus) more than doubles the odds that a student who
enters postsecondary education will complete a bachelor's degree.
Propose a study design that might enable them to able to draw this conclusion.
Discuss why it would be difficult and probably unethical to perform that study.

Think about the gray areas involved in the following?

Is it okay to use animals for experiments? Does it matter whether we are trying to develop treatments
that may potentially save human lives, or just to develop cosmetics that wont damage peoples eyes?
Under what conditions would people consent to try an unapproved experimental treatment? What is
informed consent? If you have been told that you have but months to live, would you volunteer for an
experiment? Would you want to be part of the treatment group or get the placebo?
Is it okay to determine whether mastectomies are an effective treatment for breast cancer by randomly
deciding which patients have mastectomies and which a less disfiguring procedure?
Is it acceptable to drill placebo holes in patients skulls so that they and their doctors wont know
whether or not they received an experimental implantation of nerve cells as a possible treatment for
Parkinsons disease? (Yes, that experiment was approved.)

Finally, I thought this story about one of the authors was interesting enough to put in the
notes so you could read it at your leisure. I know, I know, like you dont already have
enough things to read for that crazy English teacher of yours.
The Salk polio vaccine trials.
Ask your parents (or grandparents) about the polio epidemics of the early and mid-20th century.
Imagine a disease that strikes children, killing many and crippling others. Parents were afraid, with just
cause.
Parents took many desperate measures to try to prevent their children from contracting polio. Because
the epidemics peaked in the summer, some blamed them on swimming, so children were forbidden to go
swimming. One conjecture linked the disease to peach fuzz, so children were forbidden to eat fresh
peaches.
Amidst this climate of fear, someone seeks parental approval to try an experimental vaccine in the hope
of preventing the disease. This vaccine involves injecting a small dose of polio virus into your child. If
you were a parent, would you give permission?
In 1955, one of the authors was a subject in this experiment. Randomly chosen school districts sought
permission from parents to use their second graders as subjects. Those whose parents agreed were
transported to the local vaccination site in the spring. Imagine a long line of kids, inching forward to get
a shot. We stood there, watching and listening as each child in turn reached the front of the line and
cried when the vaccine was administered. We waited for our turn to cry. About a month later, we made
the trip again for the second dose. Another line, another turn to cry. After the summer polio season, the
figures were checked; it was clear that the incidence of polio was dramatically reduced among those
who got the vaccine compared to those who received a placebo injection of a saline solution. The federal
government mounted a nationwide program to vaccinate all children, and in a few years the disease was
conquered.
Now imagine being 7 years old and being told in October that the shots you got last spring were fake,
so now you have go get the real ones. Your interest in experimental design is further piqued (no pun
intended) when you learn that your best friend doesnt need any more shots because he got the real ones
the first time. (So, eventually you write a Statistics textbook.)
Think more about the ethics here. Some children who got the placebo shot died or were crippled by the
disease when the vaccine might have saved them. But how could that be known in advance? And if
some doses of the vaccine were a little too live, they may have actually caused the disease in children
whose parents gave permission in hopes of protecting them.
Perhaps one of the most interesting sidebars is that the experiment actually had a second design,
approved by some schools, in which all the children whose parents approved got vaccinated and the
others were used as controls. Surprisingly, the results of this design were not as convincing. It seems that
parents with higher levels of education were more likely to give permission, and also likely to raise their
children in more hygienic conditionswhich put those children more at risk of polio! Children reared in
less hygienic surroundings were more likely to contract a mild form of polio early in life, thus
developing a natural immunity to the disease when they were older. This rendered the treatment and
control groups not identical at the start of the experiment, confounding the results.
Moral: Theres no substitute for double-blinded, placebo-controlled, randomized clinical trials.