Evidence-based Medicine

The randomized controlled trial: A graphic framework

Facilitator: Dr Rohan Maharaj 2011

Objectives
To provide a simple framework for remembering the RCT The student will have an easy way of understanding where biases can affect the RCT To learn about critically appraising a paper based on a RCT

Competencies
Create a PICO question on therapy from a given clinical scenario List the database resources of full-text papers on RCTs Search, identify and retrieve a full-text paper using the RCT methodology Critically appraise an RCT Defend your clinical decision based on the previous steps

Competencies
List and describe the major elements of the RCT Conduct the basic statistical analysis of the results of an RCT Describe the common biases and list the potential weaknesses of the RCT Use all the above in your final decision on the quality of the RCT and whether or not you will incorporate the information into your practice or reject the results of the RCT

Developing a question for a clinical intervention

Patient Intervention Comparison Outcome

Tips for How would I building describe a similar patient of mine? Example: In a 18 year old female with exposure to chickenpox

Which intervention am I considering

What is the What can I main alternative accomplish? to compare with the alternative? lead to preventing an attack of chickenpox in the 18 year old

..does the .compared with use of a no vaccine.. chickenpox vaccine.

Locating
Therapy Cochrane Systematic reviews No Results DARE
Control Trials Register No Result
Other type of question

PubMed
Primary Search
DARE: Database of Abstracts of Review of Effects

Clinical Queries Prognosis Etiology Therapy Testing

Practice guidelines

MeSH terms

Steps in a RCT
1. 2. 3. 4. 5. Sampling and selection Randomization Application of intervention(s) / placebo Outcomes Analysis

Flow diagram for a RCT

4 Step 1 2
3

Step 1: Selection and sampling issues

What sort of patients were included? Were they from PC or a Referral centre? Do the exclusion and inclusion criteria make sense? Were consecutive patients recruited? These considerations affect the external validity and generalisability of the study.

Flow diagram for a RCT: Internal validity

Internal validity External validity

1 2 Exclusions
3

Step 2: Randomization
Are the baseline characteristics of the study groups similar? If the groups are small, then there is the risk that discrepancies occurred by chance. Concealment of allocation

Flow diagram for a RCT: Sources of Bias

Internal validity External validity
Cross-over

1 2 Exclusions
3

4
Contamination

Co-intervention

Count/ Loss to follow-up

Step 3: Sources of bias in RCTs

Co-intervention Cross-over Contamination Compliance Count (Loss to follow-up)

Contamination
Contamination-The inadvertent application of the intervention to members of the control group, or inadvertent failure to apply the intervention to the members of the experimental group. E.g. control and experimental patients share their medications or both groups end up having an educational intervention.

Crossover
Occurs when both patients and physicians know what medication the patient is receiving and also know what are the alternative treatments. E.g. when patients in a less aggressive treatment arm crossover into a more aggressive treatment.

Co-Interventions
Introduce bias if they are applied differentially to the trial arms- E.g. In a trial of Daflon 500 for treatment of venous ulcers, adjunctive treatments such as elevation of the limb was not described. Could some patients have had elevation and others none?

Step 4: Outcomes
Are the chosen outcomes reasonable? Were all important outcomes considered? The importance of blinding Single, double & triple blinding

Step 5: Analysis
Intention to treat analysis Magnitude of effect- CER, EER, RR, RRR, ARR, NNT, OR, precision (confidence levels), and sub-group analysis

Step 5: Analysis- Count

Intention-to-treat:

Every participant analysed according to his randomised group regardless of whether he received the assigned intervention or not for what ever reason. Preserves the value of randomisation.
May underestimate full effect of the treatment, but guards against more important causes of biased results.

Intention to Treat Analysis: An example

Control
30 lost

1000

970

35 patients die /970=3.6%

1000

960

20 patients die /960= 2.1%

Intervention

40 lost

Intention to Treat Analysis: An example

Control
30 lost ITT ANALYSIS: Assume patients alive 35/1000=3.5%

1000

970

35 patients die /970=3.6%

1000

960

20 patients die /960= 2.1%

Intervention

40 lost

ITT ANALYSIS: Assume patients died 60/1000=6.0%

Step 5: Analysis- Compliance

Per protocol analysis:

Includes only those participants who took a certain proportion of the intervention/completed a certain proportion of visits etc. But, participants who adhere may be different from drop-outs in ways that are related to the outcome of interest.

Some trials evaluate results using both methods.

Similar results increase the confidence in the conclusions of the trial.

Step 5: Analysis
Subgroup analyses:
-Comparisons

between randomised groups in a subset of the trial cohort.

Prone to misleading results: -Subgroups are smaller, thus there may not be sufficient power to find important differences. -When based on postrandomisation factors, they do not preserve the value of randomisation, often producing misleading results.

Step 5: Analysis: size of treatment effect

(Simplest case: 2x2 table)

Outcome Yes
No Risk of outcome

Intervention

Control

Total

r
Risk with therapy Y

s
Baseline risk X

Y= p/(p+r)

X = q/(q+s)

Relative Risk (RR): Y/X

Relative risk reduction (RRR): [(1-RR)]*100 or [ (X-Y)/X] *100

Step 5: Analysis
Absolute Risk Reduction: X-Y
Odds Ratio:

Relative risk of outcome in intervention Relative risk of outcome in control

The FIT trial

The reduction of hip fractures was 49% after 36 months follow up.

The FIT trial

The Fracture Intervention Trial (FIT) of 2027 postmenopausal women at 11 centers in the US, aged 55-80, with BMD 0.68g/cm and with previous XRay evidence of vertebral fracture, studied the effect of aledronate in preventing a fracture from occurring in the hip.

The FIT trial: Results

Placebo
Hip fracture 22

Intervention Total
11 33

No Fracture
Total

983
1005

1011
1022

1994
2027

The FIT trial: Results

Control event rate (CER) of 22/1005 (2.19%) Experimental event rate (EER) of 11/1022 (1.08%) The RR= 1.08/2.19, approximately 49%. RRR= 100 - RR/100 = 51%

Relative risk (RR) or relative risk reduction (RRR).

RR and RRR can lead to false expectations among clinicians and patients regarding the potential impact of the treatment in individual patients.
RR can appear large even if the event rates in the RCT are small

The FIT trial: Results

The absolute risk reduction (ARR) was CER-EER= 2.19%-1.08%=1.11%. The NNT is 100/1.11= 90. That is we need to treat 90 patients for 36 months to prevent one additional fracture. COPE: the cost to a health system to prevent one fracture is 90x$10 (TTD) x365x3=$985 500.

Hypothetical studies measuring the results of placebo vs. treatment

CER EER RRR NNT

TRIAL A 50%

25%

50%

TRIAL B 0.02%

0.01%

50%

10 000

Step 5: Analysis: Precision

Relative risk, Relative Risk Reduction,Odds Ratios:

Point estimate. 95% Confidence Interval (CI).

95% 0

Step 5: Analysis: Precision

Effect of sample size: as sample size C.I. becomes smaller & ones confidence in the results .
Positive study conclusion that treatment is effective. Examine lower limit of C.I. of RRR is it acceptable for you? Negative study conclusion that treatment in not more effective. Examine upper limit of C.I. OF RRR is it acceptable for you? Calculate 95% CI of RR or RRR if Standard Error (SE) of RR or RRR is given by: 95% CI = RRR (or RR) [2*SE (RRR or RR)]

Step 5: Analysis
NNT- number need to treat 1/ARR The number of patients you need to treat to prevent or promote one additional event.

Limits
A continuous outcome e.g. BP or pain scale Comparison of old treatment vs. new

Critical appraisal of a randomized controlled trial investigating a new therapy or comparing a new therapy with an older one

3 Headings
Are the results of the study valid?
What were the results?

Will the results help me in caring for my patients?

Are the results of the study valid?

Primary guides Was the assignment of patients to treatments randomized? Were all patients who entered the trial accounted for and attributed at its conclusion?

Are the results of the study valid?

Secondary guides Were patients, health workers, and study personnel blind to treatment? Were the groups similar at the start of the trial? Aside from the experimental intervention were all groups treated equally?

What were the results?

How large was the treatment effect? How precise was the estimate of the treatment effect?

Will the results help me in caring for my patients?

Can the results be applied to my patient care? Were all the important clinical outcomes considered? Are the likely treatment benefits worth the potential harms and costs?

Conclusion
RCT Graphic model Critical appraisal criteria for assessing a paper which employs a RCT

References
Attia J and Page J. A graphic framework for teaching critical appraisal of randomized controlled trials. EBM May/June 2001;6:68-69. Maharaj RG. Developing an evidencebased Caribbean practice: NNT. Postgraduate Doctor (Caribbean); March/April 2005;21(2):36-42.