Evaluating the Role of Bias

Definition of Bias
Bias is a systematic error that results in an
incorrect (invalid) estimate of the measure of
association
A. Bias can create spurious association when
there really is none (bias away from the null)
B. Bias can mask an association when there
really is one (bias towards the null)
C. Bias is primarily introduced by the
investigator or study participants
Bias slide #2

Definition of Bias (contd)

D. Bias does not mean that the investigator is
prejudiced.
E. Bias can arise in all study types: experimental,
cohort, case-control

F. Bias occurs in the design and conduct of a

study. It can be evaluated but not fixed in the
analysis phase.
G. Two main types of bias are selection and
observation bias.
Bias slide #3

Selection Bias
A. Results from procedures used to select
subjects into a study that lead to a result
different from what would have been obtained
from the entire population targeted for study
B. Most likely to occur in case-control or
retrospective cohort because exposure and
outcome have occurred at time of study
selection
Bias slide #4

Selection Bias in a Case-Control Study

A. Occurs when controls or cases are more

(or less) likely to be included in study if
they have been exposed -- that is,
inclusion in study is not independent of
exposure

Bias slide #5

Selection Bias in a Case-Control Study

B. Result: Relationship between exposure and
disease observed among study participants is
different from relationship between exposure and
disease in individuals who would have been
eligible but were not included.
The odds ratio from a study that suffers from
selection bias will incorrectly represent the
relationship between exposure and disease in the
overall study population
Bias slide #6

Question: Do PAP smears prevent cervical cancer? Cases

diagnosed at a city hospital. Controls randomly sampled
from household in same city by canvassing the
neighborhood on foot. True relationship:

Had PAP
smear
Did not
have PAP
smear
Total

Cervical
Cancer
Cases
100

Controls

150

100

250

250

150

OR = (100)(100) / (150)(150) = .44 There is a 54% reduced risk of

cervical cancer among women who had PAP smears vs. women
who did not. (40% of cases had PAP smears versus 60% of controls)

Recall: Cases from the hospital and controls

come from the neighborhood around the
hospital.
Now for the bias: Only controls who were at
home at the time the researchers came
around to recruit for the study were actually
included in the study. Women at home were
less likely to work and less likely to have
regular checkups and PAP smears.
Therefore, being included in the study as a
control is not independent of the exposure.
Bias slide #8

The resulting data are as follows:

Had PAP
smear
Did not
have PAP
smear
Total

Cervical
Cancer
Cases
100

Controls

150

150

250

250

100

OR = (100)(150) / (150)(100) = 1.0

There is no association between PAP smears and the
risk of cervical cancer. Here, 40% of cases and 40% of
controls had PAP smears.
Bias slide #9

Ramifications of using women who were at

home during the day as controls:
These women were not representative of
the whole study population that produced
the cases. They did not accurately
represent the distribution of exposure in
the study population that produced the
cases, and so they gave a biased estimate
of the association.
Bias slide #10

Selection Bias in a Cohort Study

Selection bias occurs when selection of

exposed and unexposed subjects is not
independent of outcome (so, it can only
occur in a retrospective cohort study)

Bias slide #11

Selection Bias in a Cohort Study

Example:
A retrospective study of an occupational exposure
and a disease in a factory setting.
The exposed and unexposed groups are enrolled on
the basis of prior employment records.
The records are old, and many are lost, so the
complete cohort working in the plant is not available
for study.
If people who did not develop disease and were
exposed were more likely to have their records lost,
then there will be an overestimate of association
between the exposure and the disease. Bias slide #12

True relationship, if all records were

available
Diseased NonTotal
diseased
Exposed 50

950

1000

Un50
exposed

950

1000

RR = (50/1000) / (50/1000) = 1.00

Bias slide #13

200 records were lost, all among exposed who

did not get the disease
Diseased Non-diseased Total

Exposed

50

750

800

Un-exposed 50

950

1000

RR = (50/800) / (50/1000) = 1.25

If more records were lost in this category (exposed
subjects who did not get the disease), the bias would be
even greater.
Bias slide #14

Selection Bias: What are the solutions?

Little or nothing can be done to fix this bias
once it has occurred.
You need to avoid it when you design and
conduct the study by, for example, using the
same criteria for selecting cases and
controls, obtaining all relevant subject
records, obtaining high participation rates,
and taking in account diagnostic and referral
patterns of disease.
Bias slide #15

Observation Bias
An error that arises from systematic
differences in the way information on
exposure or disease is obtained from the
study groups
Results in participants who are
incorrectly classified as either exposed or
unexposed or as diseased or not
diseased
Bias slide #16

Observation Bias
Occurs after the subjects have entered
the study
Several types of observation bias: recall
bias, interviewer bias, loss to follow up,
and differential and non-differential
misclassification
Bias slide #17

Observation Bias
Recall bias - People with disease
remember or report exposures differently
(more or less accurately) than those
without disease.
Can result in over- or under-estimate of
measure of association.
Bias slide #18

Observation Bias

Solutions: Use controls who are

themselves sick; use standardized
questionnaires that obtain complete
information, mask subjects to study
hypothesis

Bias slide #19

Classic recall bias:Cases underreport exposure

TRUTH
Case Control

OBSERVED
STUDY DATA
Case Control

Exposed

40

20

Exposed

30

20

Unexposed

60

80

Unexposed

70

80

Total

100

100

100

100

Odds Ratio:
2.7

Odds Ratio:
1.7

Observation Bias
Interviewer bias - Systematic difference
in soliciting, recording, interpreting
information.
Can occur whenever exposure
information is sought when outcome is
known (as in case-control), or when
outcome information is sought when
exposure is known (as in cohort study).
Bias slide #21

Observation Bias
Interviewer bias
Solutions: mask interviewers to study
hypothesis and disease or exposure
status of subjects, use standardized
questionnaires or standardized methods
of outcome (or exposure) ascertainment

Bias slide #22

Observation Bias
Loss to follow up - A concern in cohort and
experimental studies if people who are lost to
follow up differ from those that remain in the
study.
Bias results if subjects lost differ from those that
remain with respect to both the outcome and
exposure.

Solution: Since that information cannot be known,

you must achieve high and equal rates of follow
up for the exposed and unexposed groups.
Bias slide #23

Observation Bias
Misclassification - Subjects exposure or
disease status is erroneously classified.
Two types of misclassification: non-differential
and differential. We will cover only the more
common form: non-differential
misclassification.

Bias slide #24

Observation Bias
Non-differential misclassification
Inaccuracies with respect to disease
classification are independent of exposure.
Or, inaccuracies with respect to exposure are
independent of disease. Will bias towards the
null if the exposure is has two categories.
Non-differential misclassification makes the
groups more similar.
Bias slide #25

Observation Bias
MisclassificationExample: Study of vaginal spermicides and
congenital disorders (Jick et al., 1981).
Solutions: Use multiple measurements,
most accurate source of information

Bias slide #26

When interpreting study results,

ask yourself these questions
Given conditions of the study, could bias
have occurred?
Is bias actually present?
Are consequences of the bias large enough
to distort the measure of association in an
important way?
Which direction is the distortion? Is it
towards the null or away from the null?
Bias slide #27