Causality

Operations Research for Managers of Reproductive Health Programs

2008 The Population Council, Inc.
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Causality
when one event produces a second event
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Why Study Causality?
We are making program decisions to
maintain, expand or discard the intervention
and need evidence that the intervention,
rather than some other factor, caused the
observed change.
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Causality Requirements
A (intervention) precedes B (outcome)
B is present only when A is present
We can rule out all other possible causes
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
The Basic Experimental
Principle
The intervention is the only difference between
set of observations/groups
This is achieved by random assignment
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
The Gold Standard
An experimental design with random assignment
is the gold standard for demonstrating causality
Required by FDA to license drugs
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Symbols
O = Observation
X = Intervention
RA = Random Assignment
Independent Variable = X
Dependent Variable = O
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Basic Experimental Design
experimental group: O1 X O2
RA
control group: O3 O4
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Characteristics of an
Experimental Design
Manipulation of intervention
Comparison of experimental and control groups
Control of threats to validity
- Random assignment
- Matching
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Random Assignment
Random sampling: In surveys, random
sampling ensures that individuals in the study
are truly representative of the population from
which they are drawn.
Random assignment: In experiments, random
assignment ensures that the experimental and
control groups are truly comparable.
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Matching
OR often uses small numbers of experimental units
Matching improves the equivalence of small groups
Units are matched on an initial measure prior to the
experiment
Is done prior to random assignment
Is done purposively (researcher selects factor on
which pairs will be matched)
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Limitations of Random
Assignment
Random assignment does not protect against
bias caused by:
Contamination
Very small number of units
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Units of Study in OR
Units of study in OR can be individuals: clients,
providers, the general public
Units of study can be groups: facilities
(hospitals, wards, clinics) villages, districts
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Causal Comparisons
O1 = O3
O1 O2
O2 O4
O3 = O4
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Post-test Only Control Group
Design
X O
RA
O
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Comparison of Post-test/Pre-
test with Post-test Only Design
Post-test Only is cheaper and quicker
All else being equal, post-test/pre-test
designs are preferred:
- Allow us to measure amount of
change
- Permits matching
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Two Multiple Group Designs
(Random Assignment)
O X O
O Y O
O O
Also:
O X O
O Y O
O X+Y O
O O
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Multiple Group Designs
An efficient way to test more than one solution
Best design to study integration of services
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Statistics
Used to determine if differences between
groups are greater than could be expected
by chance alone
Are the differences real?
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Statistical Significance and
Practical Importance
Statistical significance does not imply that
the result is important.
A conservative approach to interpreting
data:
If result is not statistically significant
STOP
If result is statistically significant ask is
the difference PRACTICALLY
IMPORTANT?
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Commonly Used Statistics
Cross-tabulation
Chi Square (compares distributions-used with
cross-tabs)
Z-test (compares 2 proportions)
T-test (Compares 2 means)
F-test (Compares more than 2 means)
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Statistical Significance
States the probability that difference between groups is
greater than chance (real)
Function of sample size and strength of intervention
Conventions:
p<.05 = 1 chance in__ that the difference is not real
p<.01 = 1 chance in__ that the difference is not real
p<.50 = 1 chance in__ that the difference is not real
Operations Research for Managers of Reproductive Health Programs
2008 The Population Council, Inc.
Operational Definitions
Terms and variables must be defined in a way that
permits measurement and monitoring.
No: The independent variable is group
counseling
Yes: Groups < 8 persons meet 2 hrs/day for 3
consecutive days. Topics include What is HIV?
(45 minutes).