Prognosis critical appraisal guide

Validity Where do I look?

Are the objectives clearly stated?

• Title, abstract, or
The main question being addressed should be clearly stated. The question will often final paragraph
be expressed in terms of a simple relationship but may not fit a PICO question. of the
introduction

Were objective and unbiased outcome criteria used?

• Methods
This is simpler with clear outcomes like death. • Statistical
methods
Where the outcome requires clinical judgement (e.g. assessment of disability) then
criteria for that outcome should be well defined and validated, and ideally assessors
should be blinded to whether the patient had the potential prognostic factor.

Was there adjustment for important prognostic variables?

• Methods
If the prognosis of two groups of patients is compared, their clinical characteristics • Statistical
should be similar at baseline, or the analysis should include statistical adjustments methods
for prognostic variables.

Adjustment should also be made for treatments which may confound the study
outcomes.

Did the study have a sufficiently large sample size?

• Methods
Larger samples usually mean more precise results. • Statistical
methods
An online sample size calculator is available at:
http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSize.

Was the sample of patients representative, well-defined and a similar point in the
disease? • Patients
• Methods
Look at how the target disorder was defined in the inclusion criteria. Did they select
the sample from a tertiary hospital setting or the community? Are all patients at the
same point in the disease? Duration of disease may relate to prognosis.

Was follow-up sufficiently complete and was it long enough?

• Methods
A flow diagram may show loss to follow up. Was the follow up time long enough for • Results
the outcome they were looking for to occur? Loss to follow up may not be random
and may relate to either a good or a bad outcome.

A ‘sensitivity analysis’ can investigate the effects of loss to follow up. For example,
the investigators may re-analyse the data assuming that all patients lost to follow up
died. This can show how the ‘worst case scenario’ would affect the results.

Faculty of Medicine, Dentistry and Health Sciences UWA Medical and Dental Library
Clinical Importance Where do I look?

How have the outcomes over time been quantified?

• Results
May be expressed as a likelihood of an event in x years, months or weeks, an odds
ratio, relative risk or displayed as a ‘time to event’ curve (Kaplan Meier Curve).

A Kaplan Meier curve shows how the chance of the event occurring changes over
time. A survival curve is one type of time to event curve but time to any event may
be graphed. See ‘Resources – Advanced concepts’ for more information on Kaplan
Meier curves.

How precise are the estimates of likelihood?

• Results
A 95% confidence interval should be given so that the reader can understand the
precision of the result.

Are the results discussed in relation to existing knowledge and is the discussion
biased? • Discussion

The discussion should place the results into a clinical context and the authors
conclusions should be justified by the study results.

What level of evidence does this paper give?

• Methods
Try to assign a level of evidence using the Oxford CEBP Levels of Evidence Hierarchy.

Applicability Where do I look?

Were the study patients and their management similar to those in my practice?
• Methods
Decide if you can generalise the study population to your patient. Would your (inclusion and
patient have been included in the study? Are they in the same setting? Do they have exclusion
the same severity of disease? criteria)

Are the results useful in patient management in your practice?

• Consider your
Does this paper answer your clinical question or have you changed your question to patient and
suit the available literature? practice

Look for important geographical differences in the disease. Consider whether your
control treatment would be the same as the study control arm. Ask yourself
whether your patients present at the same disease stage.

How would I clearly express the results to a colleague or my patient?

• Results
Try to extract data that helps you to describe the study findings to a patient or
colleague in plain English. Can you perform EBP calculations to help you do this? Put
a NNT, ARR, RRR, or NNH or survival figure into a sentence for your patient.

More information about Relative Risk, Absolute Risk Reduction, and Number Need
to Treat (harm).

Faculty of Medicine, Dentistry and Health Sciences UWA Medical and Dental Library