What is Experimental

Research
Experimental research, is
characterized by controlled
manipulation of an active
independent variable or variables by
researchers.
This controlled manipulation can be
used with groups of participants or
with individuals.
 Remember
Group Refers to treatment of Data.

Treatment may be administered to

one participant at a time, but data
from those individuals are grouped
for analysis.
 ASSUMPTIONS OF GROUP
DESIGNS
One of the major concerns of
experimentation, particularly with humans,
is the variability that appears in data.
Researchers will not find any reported
experiment, in any body of literature, in
which all human subjects produced the
same results.
What are the causes of this variability?
 ASSUMPTIONS OF GROUP
DESIGNS
It is difficult to control all the factors that
might lead to experimental results
coming from something other than
manipulation of independent variables.
Group experimental designs assume that
variability is intrinsic to human subjects,
rather than coming from sources external
to the people involved.
 ASSUMPTIONS OF GROUP
The
DESIGNS
countless factors could fill a pageage differences,

intelligence, motivation, health status, and so on.

The critical point is that these are all factors that experimenters

cannot control very well, if at all.

That leads to the notion that, if an experimenter uses a large

enough sample and the sample is randomly drawn from the

population and randomly assigned to the groups in the research,

all those intrinsic factors will be balanced out between

experimental and control groups, and the only effects left (aside

from experimental errors) will be those of the independent

variable or treatment.
 ASSUMPTIONS OF GROUP
DESIGNS
Using this process, it becomes clear that,
because emphasis is on the data of each of
the groups in an experiment, rather than the
individuals in the groups, experimenters must
compare the averages of the groups and that
differences between group averages are
derived by application of inferential statistics.
 ASSUMPTIONS OF GROUP
DESIGNS
The underlying assumption is that the statistically significant

differences will represent true differences between or among

groups, at least to a satisfactory degree of probability.

To apply these assumptions, a further assumption must be

that it is possible to create truly random samples of a

population (to represent the population of interest) and to

truly randomly assign participants to groups (to ensure the

pre-experimental equivalence of groups on dependent

variables of interest).
 ASSUMPTIONS OF GROUP
DESIGNS
A final assumption of group
experimental designs, based on
random selection and assignment, is
that results of a valid experiment can
be generalized to the population as a
whole.
 RANDOMIZED CONTROLLED
TRIALS
(RCT)can provide strong evidence for one particular research

purpose: determining whether clinical interventions work.

RCTs are prospective and experimental, meaning that

1. the independent (treatment) variable is subject to the controlled manipulation

of the investigator and

2. the dependent (measurement or outcome) variables are collected under

controlled conditions.

The independent variable consists of at least two levels,

including a treatment group and some form of comparison, or

control, group.

The randomized component of the RCT refers to random

allocation of participants to treatment and control groups. RCTs

 By definition then, all RCTs are experimental

designs.

However, not all experimental designs are RCTs,

in that there are many non-random ways of

placing participants into groups.

Although the RCT is considered a strong form of

evidence about the effect of an intervention,

RCTs are not appropriate to answer all questions,

nor is every topic of study ready for RCTs.

 Example
For example, if one wanted the strongest possible evidence

about whether cigarette smoking causes lung cancer, one

would design and implement an RCT comparing a smoking

group to a non-smoking group.

Given the strength of evidence from non-randomized trials,

would it be ethical to randomly assign a large group of non-

smoking adolescents to smoking and non-smoking groups

and follow them throughout their lives to determine the

relative rates at which they develop lung cancer? Clearly not.

 Randomization
In random selection, all members of the
defined population are available and
willing to participate in the study.
Further, random sampling assumes that
all members of a population have an
equal chance of being selected for a
study.
 Issues while Doing
Randomization
There are at least two roadblocks to the true random selection.

First, it is very difficult, if not impossible, to identify all

members of a clinical population.

For example, in research on therapy for severe ankle sprain,

many members will not be identified because they did not seek
medical services and, thus, there is no record of their having
sprained ankles.

Not identifying all members of a population means that all

members did not have an equal chance of being selected.
 Issues while Doing
Randomization
Second, because of present-day research ethics, researchers can

only use participants who volunteer.

Thus, all participants in clinical research are self selected.

Self-selected participants may not truly represent the population.

For example, in occupational therapy for handwriting after hand

surgery, only those with the most severe degrees of writing deficit

might seek treatment.

Once we see that the selected group might not represent the

population, we must question the inferential generality of an

experiments results to the population at large.

 Problems with RCTs
RCTs are fraught (troubled) with enormous practical
difficulties, such as finding and matching large samples of
relatively homogeneous participants.

In RCTs, patients are (perhaps) randomly assigned to fixed

treatment conditions. However, intervention is
individualized.

There is no assumption that results of a group-design

experiment are generalizable to any particular individual,
because results are based on averages and the average
person probably does not exist.
SINGLE-FACTOR EXPERIMENTAL
DESIGNS
Single-factor experimental designs
have one independent variable.
 PretestPosttest Control-Group
Design
One of the classic RCT designs.

The design calls for at least two groups randomly selected (as well as

possible) from the population of interest and randomly assigned to

one group or another.

Performance on the dependent variable(s) is measured and, often,

equivalence on the dependent variable(s) is determined statistically.

Typically, when there are two groups, one group will then receive

treatment (the independent variable) and the other group will not.

At the end of the treatment period, performance on the dependent

variable(s) is measured again.

 PretestPosttest Control-Group
Design
ROXO

RO O

The R indicates that participants were assigned

randomly to the two groups.

The Os represent observation, or measurement.

The X represents an intervention, or manipulation.

The blank in the second row represents the control

group.
 The intervention and the control, therefore, constitute the two

levels of the independent variable.

Another term for this type of design is a between-subjects

design or a between-groups design because the differences of

interest are those differences that occur between subject

groups.

This may also be referred to as a parallel group design,

wherein each group receives only one of the levels of the

independent variable.

The control group in this design may be referred to as a

passive control group because it receives no treatment at all

or else receives a sham or placebo treatment.

 Example
Runeson and Hake used the classic pretestposttest control-group design to

study the effect of iontophoresis with cortisone on the treatment of lateral

epicondylitis, or tennis elbow.

They compared two groups: a treatment group that received cortisone

iontophoresis and a passive control group that received sham

iontophoresis.

They were treated four times over 2 weeks and were measured at the

conclusion of the treatment and at 3 and 6 months following the

completion of the treatment.

Both groups improved during the study and follow-up period, and there was

no statistically significant difference in the response between the two

groups, leading the authors to question the use of cortisone iontophoresis

in the treatment of tennis elbow.

 Clinical Alterations
In clinical research, the pretestposttest control group design is often
altered slightly, as follows:

R O X1 O

R O X2 O

This alteration is made when the researcher does not believe it is

ethical to withhold treatment altogether or when two treatments are
known to be effective and the researcher wishes to compare their
effectiveness.

In this version of the RCT, one group receives a typical treatment and
the other group receives the experimental treatment.

A control group receiving a standard treatment can be referred to as

an active control group.
 Other Variations
Additional variations on the classic pretestposttest control-

group design include taking more than two measurements

and using more than two treatment groups.

The general notation for this design would be as follows,

with the appropriate number of groups and measurement

periods:

R O O X1 O O

R O O X2 O O

ROO OO
 Posttest-Only Control-Group
Design
Researchers use this design when they are not
able to take pretest measurements.

Therefore, the posttest is the only basis on which

to make judgments about the effect of the
independent variable on the dependent variable.

Like the pretestposttest control-group design,

this design is an RCT and a between-groups
design.
Posttest-Only Control-Group
Design
RXO
R O
Single-Group PretestPosttest
Design
OXO
 Nonequivalent Control-Group
Design
The nonequivalent control-group design is used when a
nonrandom control group is available for comparison.

OXO

-----------

O O

The dotted line between groups indicates that

participants were not randomly assigned to groups.

Thus, this design is not considered a randomized

controlled design.
 Time Series Design
The time series design is used to establish a
baseline of measurements before initiation of
treatment to either a group or an individual.
When a comparison group is not available, it
becomes important to either establish the
stability of a measure before implementation of
treatment or document the extent to which the
measure is changing solely as a function of time.

OOOOXOOOO
Repeated Measures or Repeated
Treatment Designs
Repeated measures designs are widely used in health
science research.

The term repeated measures means that the same

participants are measured under all the levels of the
independent variable.

In this sense, any of the pretestposttest designs

can be considered a repeated measures design because
each participant is measured at both levels (before and
after treatment) of a possible independent variable.
 RCT with a Cross over
Design
The repeated treatment design is a type of repeated
measures design in which each participant receives more
than one actual treatment.

The repeated measures designs are also referred to as

within subjects or within-group designs because the effect
of the independent variable is seen within participants in a
single group rather than between the groups.

When the order in which participants receive the

interventions is randomized, this design is considered an
RCT with a cross-over design.
 You should be aware of some factors that limit the effectiveness of repeated

measures designs, particularly when there are more than two treatments or more

than two levels of an independent variable.

Primarily, designs in these cases are subject to order effects and carryover effects.

Because there are multiple treatments, the order in which they are presented may

influence the outcomean order effect.

It is critical to randomize the order to each participant so that each level or

treatment appears at least once in all possible positions.

Further, the researcher must ensure that each treatment is administered to an

equal number of participants.

Similar to the order effect is the carryover effect. In this case, it is possible that a

treatment (or level of an independent variable) has a specific effect on the

treatment that follows it, either enhancing or diminishing performance.

Again, the cautious researcher will vary the sequence of treatments to minimize the

possibility of this effect.

 There are two basic strategies for selecting treatment
orders in a repeated measures design with several
levels of the independent variable.
These two strategies can be illustrated by a
hypothetical study of gait characteristics with four
different foot units in a transtibial prosthesis:
a solid-ankle-cushion-heel foot (SC),
a single-axis foot (SA),
a multiaxial foot (MA), and
a dynamic response foot (DR).

The first strategy is to randomly select an order and

then rotate the starting position.
 In a random start with rotation, each
condition appears at each position in
the rotation equally often.
One fourth of the participants would be
randomly assigned the order in the
first row; one fourth, the order in the
second row; and so on.
 The Latin square technique ensures not only that
each condition appears at each position equally
often, but also that each condition precedes and
follows every other condition equally often.
Thus, a Latin square has a greater level of
randomization than does a random start with
rotation.
MULTIPLE-FACTOR EXPERIMENTAL
DESIGNS
 Questions That Lead to a
Multiple-Factor Design
Researchers usually design multiple-factor experiments
because they are interested not only in the individual
effects of the multiple factors on the dependent variable,
but also in the effects of the interaction between the
multiple factors on the dependent variable.

For example, say we wanted to conduct a study to

determine the effects of different rehabilitation programs
on swallowing function after cerebral vascular accident.
 We could start by selecting 60 patients for study
and randomly assigning them to one of the three
groups
(posture program,

oral-motor program, and

a feeding cues program)

The first independent variable would therefore be

type of treatment, or group.
Then assume that two different therapistsperhaps
a speech-language pathologist and an occupational
therapistare going to provide the treatments.
 We might wonder whether one therapist might simply get better

results than the other therapist, regardless of the type of

treatment provided.

A second independent variable, then, is therapist.

If therapist is added as a second independent variable, a third

question must also be asked in this design: Is one therapist more

effective with one type of treatment and the other therapist more

effective with another type of treatment?

In other words, is there an interaction between type of treatment

and therapist?

It is the study of interaction that clearly differentiates a multiple-

factor experimental design from a single-factor design.