Pre-Medicine Program 3

Biostatistics and Epidemiology 2

Fall Semester 2015
Public Health Epidemiology II
Validity, reliability, screening
Dr Iain Blair
Institute of Public Health

Summary

What is screening
What makes a good screening test
Reliability, validity
Sensitivity, specificity
Predictive values

Learning Outcomes
1. Define screening
2. Give examples of screening test and programs
3. Explain reliability and validity in the context of screening
tests
4. Calculate and interpret sensitivity, specificity, and
predictive values.

Which of these is screening

A man aged 60 having a chest X-ray because he has noticed blood in his
sputum.
A man aged 55 having a resting electrocardiogram ordered by his
anesthetist, prior to undergoing hernia repair under general anesthetic.
A man aged 20 having a heaf test because he shares a flat with someone
just diagnosed with pulmonary tuberculosis.
A man aged 50 having a prostate specific antigen blood test because his
friend at the golf club has just been diagnosed with prostate cancer.
A well woman aged 65 having a routine mammography (breast X-ray).
A man aged 48 having a liver function blood test before commencing drug
treatment that carries risk of liver toxicity.
A newborn baby having a heelprick bloodspot test to check for
hypothyroidism.
A man aged 30 having a vision test in order to renew his Heavy Goods
Vehicle drivers license.

write down a one sentence simple explanation of

what screening is (no jargon)

Definition
Screening is a process of identifying apparently
healthy people who may be at increased risk of a
disease or condition.
They can then be offered information, further tests
and appropriate treatment to reduce their risk
and/or any complications arising from the disease
or condition.

Prevention
Primary: prevent onset of disease

Secondary: minimise consequences of disease

once it has arisen - prevent it getting worse
Tertiary: prevent death or permanent disability
once condition is established

What is screening?
Finding people at risk of a health problem
before they get symptoms
So they get earlier, potentially more effective
treatment or make informed decisions about
their health.
The identification of unrecognized disease by
the application of a simple rapid test

Benefits and adverse effects of screening

Benefits

Adverse effects

Cure
Delayed death
Better quality of life
Reassurance
Reduction in morbidity

Pain and discomfort

associated with test and
treatment
Anxiety
Adverse effects of test itself
e.g.. X-rays

For screening to make a real difference to health then

testing and intervention at a symptomless stage must
give a better outcome

Some Common Screening Tests

Pap smear for cervical dysplasia or cervical cancer

Mammography for breast cancer
Fasting blood cholesterol for heart disease
Fasting blood sugar for diabetes
Blood pressure for hypertension
PSA test for prostate cancer
Fecal occult blood for colon cancer
Ocular pressure for glaucoma
PKU test for phenylketonuria in newborns
TSH for hypothyroid and hyperthyroid

Diagnostic Test and Screening Test

A diagnostic test is used to determine the
presence or absence of a disease when a
subject shows signs or symptoms of the disease
A screening test identifies asymptomatic
individuals who may have the disease
The diagnostic test is performed after a positive
screening test to establish a definitive diagnosis

Screening delivery
Varies from one country to another.
All the elements, from identification of eligible
individuals through to treatment for screen detected
conditions, are provided as part of a single system.
Different elements are provided by different agencies,
participant to seek help first from one then from
another.
Not just a test

A good screening test

Simple
Rapid
Inexpensive
Safe
Acceptable

Correctly Classifying Individuals by Disease

Status

Tests are used in medical diagnosis, screening,

and research
How well is a subject classified into disease or
non-disease group?
Ideally, all subjects who have the disease should be
classified as having the disease and vice versa
The ability to classify individuals into the correct
disease status depends on the accuracy of the tests,
among other things

A good screening test

Reliable/precise
Accurate
Valid

Reliability

Reliability is a property of a measure that refers to

its statistical stability, or the degree to which
multiple observations of identical phenomena yield
identical results
gives the same result (negative or positive) on
repeated applications

Validity

Validity is a property of a measurement that refers

to its accuracy, or the degree to which
observations reflect the true value of a
phenomenon.

Ability to give a true measure

Sources of unreliability
Because of variation in laboratory procedures,
observers, or changing conditions of test
subjects (such as time, location), a test may not
consistently yield the same result when repeated
Different types of variation
Intra-subject variation
Intra-observer variation
Inter-observer variation

Sensitivity and Specificity

Sensitivity:
the ability of a test or case definition to identify true
cases; the proportion of people with a health
condition that are correctly identified
positive in disease

Specificity:
the ability of a test or case definition to exclude
persons without the health condition of interest; the
proportion of persons without a health condition
that are correctly identified
Negative in health

Determining the Sensitivity, Specificity of a

New Test

Must know the correct disease status prior to

calculation
Gold standard test is the best test available
It is often invasive or expensive

A new test is, for example, a new screening test

or a less expensive diagnostic test
Use a 2 x 2 table to compare the performance of
the new test to the gold standard test

Gold standard test

DISEASE
Present

Absent

All people with

disease

All people without

disease

Comparison of Disease Status: Gold

Standard Test and New Test

DISEASE
TEST

Present

Absent

Positive

A
True positive

B
False positive

Negative

C
False negative

D
True negative

Sensitivity
Sensitivity = a/(a+c)
TP/TP+FN
Sensitivity is a fixed characteristic of the test

Specificity
Specificity = d/(b+d
TN/TN+FP
Specificity is a fixed characteristic of the test

Applying Sensitivity and Specificity to a

Screening Test
DISEASE
TEST

total

Present

Absent

Positive

80

100

180

Negative

20

800

820

total

100

900

1000

Calculating Sensitivity and Specificity

Sensitivity = 80/100=80%

Specificity = 800/900=89%

Examining the Effect of Changing Cut-Points

Where to Draw the Cut-Point

If the diagnostic (confirmatory) test is expensive or
invasive:
Minimize false positives or
Use a cut-point with high specificity

If the penalty for missing a case is high (e.g., the

disease is fatal and treatment exists, or disease
easily spreads):
Maximize true positives
That is, use a cut-point with high sensitivity

Balance severity of false positives against false

negatives

Exercise
The prevalence of a disease is 30%
1000 subjects are tested with a new test for the
disease
340 subjects test positive but of these 100 do
not have the disease
Draw a table, fill in the cells and calculate
sensitivity and specificity for this example

DISEASE
TEST
Positive

Negative
total

Present

Absent

total

Predictive value
Positive predictive value
the proportion of cases identified by a test that are true
cases
true positives divided by the number of true-positives
plus false-positives.
a/(a+b)
Negative predictive value
Proportion of those with negative test result that are
true negatives
True negatives divided by all negatives
d/(c+d)

DISEASE
TEST

Present

Absent

Positive

A
True positive

B
False positive

Negative

C
False negative

D
True negative

Exercise

Population 1000
Prevalence of diabetes 10%
Test: Blood sugar
Sensitivity= 80%
Specificity=89%

Prevalence=10%
DISEASE
TEST
Positive

Negative
total

Present

Absent

total

Exercise

Population 1000
Prevalence of diabetes 1%
Test: Blood sugar
Sensitivity= 80%
Specificity=89%

Prevalence=1%

DISEASE
TEST
Positive
Negative
total

Present

Absent

total

Sens=80% Spec=89% Prevalence=10%

DISEASE
TEST

total

Present

Absent

Positive

80

100

180

Negative

20

800

820

total

100

900

1000

Predictive value
Positive predictive value
80/180=44%
a/(a+b)
Negative predictive value d/(c+d)
800/820=98%

Sens=80% Spec=89% Prevalence=1%

DISEASE
TEST

total

Present

Absent

Positive

109

117

Negative

881

883

total

10

990

1000

Predictive value
Positive predictive value
8/117=7%
a/(a+b)
Negative predictive value d/(c+d)
881/883=99%

PPV Primarily Depends On

The prevalence of the disease in the population
tested

England

Northern
Ireland

Scotland

Wales

Fetal Anomaly (inc Down's Syndrome)

yes

yes

Infectious Diseases in Pregnancy

yes

yes

Sickle Cell and Thalassaemia

yes

Newborn and Infant Physical

Examination

yes

yes

Newborn Blood Spot

yes

yes

yes

yes

Newborn Hearing Screening

yes

yes

yes

yes

Antenatal and newborn

yes

yes

yes
yes
yes

Young person and adult

Abdominal Aortic Aneurysm

yes

Diabetic Retinopathy

yes

yes

yes

yes

Cancer - Breast

yes

yes

yes

yes

Cancer - Cervical

yes

yes

yes

yes

Cancer - Bowel

yes

yes

yes

Other: Prostate Cancer Risk Management, Chlamydia screening and the NHS
Health Check (for vascular risk).

The UK National Screening Committee

http://www.screening.nhs.uk/screening

module on screening
http://www.healthknowledge.org.uk/interactivelearning/screening

Now consider the FOB screening tests

You find out that your father has undertaken the
test and has a positive result
He ask you whether he has cancer?
Prevalence of disease is 0.3%
Sensitivity of 50%
False positive rate 3%.

DISEASE
TEST
Positive

Negative

total

Present

Absent

total

5%
Doctors with an average of 14 yrs experience
Answers ranged from 1% to 99%
half of them estimating the probability as 50%
Gigerenzer G BMJ 2003;327:741-744