MEASURING THE SIZE OF A TREATMENT EFFECT: RELATIVE RISK REDUCTION, ABSOLUTE RISK REDUCTION, AND NUMBER NEEDED TO TREAT

Hussein Hollands, MD, MS (epid) Peter J. Kertes, MD, CM, FRCS(C) The purpose of this review article is to outline a few concepts that are useful for interpreting the results of a randomized clinical trial (RCT). Specically, we will focus on measures of the size of a treatment effect relative risk ratio (RRR), absolute risk ratio, and number needed to treat (NNT). These concepts are important, specically with respect to decisions to implement a treatment modality in clinical practice based on the results of a RCT. To solidify these ideas, we will work through some practical examples in ophthalmology. In this manuscript, we will assume that we are interpreting results from a well-designed RCT with a clinically important dichotomous negative outcome measure (eg, 15-letter loss of visual acuity) comparing a new treatment modality with a control. clinical practice.24 We will review measures used to quantify the size of a treatment effect in the following sections. Of course, in order for a treatment modality to be considered clinically signicant, statistical signicance would have to be shown in a scientically valid manner using a clinically important outcome measure. Statistical signicance is necessary for clinical signicance but is not sufcient. It is useful to understand how sample size relates to these concepts. A larger sample size will increase the precision of the treatment effect that is observed but will not change the size of the treatment effect. A larger sample size (and thus more outcome events) leads to more condence in the result and a smaller P value or a narrower condence interval around the point estimate. In fact, any treatment effect, in theory, can be found to be statistically signicant if enough people are studied. However, even with a very large sample size, a given treatment effect (for example, the RRR) will remain similar.

Statistical and Clinical Signicance

When interpreting the results of a RCT, one must make a distinction between the statistical signicance of a result and the clinical signicance of a result. The statistical signicance refers to the precision of the treatment effect and is presented as either a P value or 95% condence interval. These results are obtained from formal statistical testing. A P value represents the probability that an observed treatment effect is due to sampling error and a 95% condence interval is a range of treatment effects in which we can be 95% condent that the true treatment effect lies.1 The clinical signicance, by contrast, refers to the size of treatment effect at which a physician feels adoption of the treatment modality would be justied in

Measures of the Size of a Treatment Effect

Most clinical trials will use a dichotomous harmful event such as a 15-letter loss of visual acuity as the primary outcome measure. The incidence rate of the primary outcome is measured in both treatment arms for the duration of the follow-up period. The size of the treatment effect versus a control is often reported as a relative risk (RR) or RRR but can also be reported as an absolute risk reduction (ARR) or NNT. Although results of a RCT are often reported with

72

complex gures and statistical analyses, it is often easiest to convert the main study results into a simple 2 2 contingency table to allow for clearer conceptualization. Table 1 shows a simple contingency table along with denitions for RR, RRR, ARR, and NNT. The RR is dened as the incidence rate of the outcome in the treatment group divided by the incidence rate of the outcome in the control group. The RRR is dened as a risk reduction (Table 1). If the treatment arm is effective in preventing disease then the RR will be less than 1.0; a smaller RR corresponds to a larger treatment effect. Again, the RR measures the size of a treatment effect and therefore does not change depending on sample size. Increasing sample size will increase the precision of the RR and thus make it more likely to be statistically signicant. Another measure of the size of a treatment effect is the ARR, which is dened as the rate of the outcome in the control group minus the rate of the outcome in the treatment group. This can be a useful and intuitive statistic as it accounts for the absolute incidence of disease.5 The ARR can also be mathematically described in an intuitive statistic called the NNT. The NNT is simply the inverse of ARR and is the number of patients that need to be treated with the intervention to prevent 1 negative outcome event. Essentially, the ARR and NNT confer the same information, except that the NNT offers superior conceptualization. As with RR, the ARR and NNT do not change with sample size, although the precision of these estimates will increase with a larger sample size.

To understand the different information that RRR and ARR confer a clinical example is warranted. The Ocular Hypertension Treatment Study (OHTS) was a landmark RCT where over 1600 subjects with ocular hypertension (intraocular pressure between 21 and 32 mm Hg) and no clinical evidence of glaucoma were randomized to either topical glaucoma medications to reach a target intraocular pressure or no treatment.6,7 The primary outcome measure was the development of glaucoma based on visual eld and optic nerve abnormalities over a minimum 5-year follow-up period. Tables 2A and B show a contingency table along with RR and ARR at 60month follow-up for the entire study population (Table 2A) and among only black patients (Table 2B).6,7 These results were simplied from survival analyses that were published. Among all patients in the study, treatment with glaucoma medication conferred a 54% RRR (Po0.001). Among only black patients, treatment conferred a 48% relative risk reduction (Po0.001). From this information alone one may conclude that the group of black subjects did slightly less well than the group of all subjects. However, black subjects were more likely to develop glaucoma than the entire patient population (12.2% vs. 6.9%) regardless of treatment. Hence, the ARR conferred from treatment was higher amongst black subjects than the entire population (7.7% vs. 5.1%). In other terms, to prevent 1 patient with ocular hypertension from developing glaucoma over 5 years, only 13 black study patients would need to be treated (NNT = 13) versus 20 patients selected from the study at large (NNT = 20).

Table 1. Generic 2 2 Contingency Table and Denitions Outcome (Harmful) Treatment group Control a c b d Total ab cd Incidence Rate (Harmful Outcome) a/a b c/c d

73

RR = Incidence outcome (tx grp)/Incidence outcome (control) = (a/a b)/(c/c d). RRR = (1RR) 100% = 100% [1(a/a b)/(c/c d)]. ARR = Incidence outcome (control)Incidence outcome (tx grp) = (c/c d)(a/a b). NNT Equation = 1/ARR = 1/[(c/c d) (a/a b)].

Table 2A. Results From the OHTS (All Patients)7 Outcome (Glaucoma Progression) Medical therapy No treatment 36 78 781 741 Total 817 819 Incidence Rate (Glaucoma Progression) 36/817 = 4.4% 78/741 = 9.5%

RR = Incidence outcome (tx grp)/Incidence outcome (control) = (a/a b)/(c/c d) = 0.46. RRR = (1RR) 100% = 100% [1(a/a b)/(c/c d)] = (1.00.46) 100% = 54%. ARR = Incidence outcome (control)Incidence outcome (tx grp) = (c/c d)(a/a b) = 9.5% 4.4% = 5.1%. NNT = 1/ARR = 1/[(c/c d)(a/a b)] = 1/0.51 = 19.6.

This example illustrates the point that the RRR looks at the risk of disease in the treatment group relative to the control group as a ratio, but does not consider the incidence of the outcome. In contrast, the ARR will increase (and hence NNT decrease) when the overall incidence of the outcome in the study population is increased. Clearly, to fully understand the size of a treatment effect and its clinical importance, both RR and ARR (and thus NNT) should be considered. Table 3 shows some clinical examples from well-cited ophthalmology trials, including the ANti-VEGF Antibody for the Treatment of Predominantly Classic Choroidal Neovascularization in AgeRelated Macular Degeneration (ANCHOR) study,8 Herpetic Eye Disease Study (HEDS I), Stromal Keratitis, Not on Steroids study,9 and the Age-Related Eye Disease Study (AREDS).10 The purpose of this table is to work through some practical examples of RR, ARR, and NNT, and is not to provide summary information from these clinical trials. Certain clinical results were used from

these studies to illustrate conceptual points rather than clinical points. The reader is referred to the study manuscripts for more details on the study results.810

Assessing Adverse Events

In addition to assessing the size of a treatment effect in a RCT, it is important to quantitatively consider complication rates with a novel therapy. A useful summary statistic is the number needed to harm (NNH), or the number of patients that would need to be exposed to a treatment modality to produce an adverse event for 1 patient. This measure can be calculated for any adverse outcome that occurs more frequently in the treatment group than the control group and is related to attributable risk. The attributable risk is simply the incidence of the adverse event in the treatment group minus the incidence of the event in the control group. Furthermore, the NNH is simply the inverse of the attributable risk. An example where NNH can be clinically applicable is in the treatment of diabetic macular edema that persists

Table 2B. Results From the OHTS Study (Black Subjects)6 Outcome (Glaucoma Progression) 186 172 Total 203 205 Incidence Rate (Glaucoma Progression) 17/203 = 8.4% 33/205 = 16.1%

74

Medical therapy No treatment

17 33

RR = Incidence outcome (tx grp)/Incidence outcome (control) = (a/a b)/(c/c d) = 0.52. RRR = (1RR) 100% = 100% [1(a/a b)/(c/c d)] = (10.52) 100% = 48%. ARR = Incidence outcome (control)Incidence outcome (tx grp) = (c/c d)(a/a b) = 16.1% 8.4% = 7.7%. NNT = 1/ARR = 1/[(c/c d)(a/a b)] = 1/0.077 = 12.9.

Table 3. Summary Statistics From 3 Major Clinical Trials in Ophthalmology Study ANCHOR* HEDS Iw AREDSz Outcome Rate (Harmful Outcome) 0.5 mg Ranibizumab: 5/139 = 3.6% Control: 51/143 = 35.7% Antiviral Steroids: 13/49 = 26% Antiviral Only: 41/57 = 73% Antioxidants Zinc: 0.202 Placebo: 0.278 RR 0.10 0.36 0.726 RRR(%) 90 64 27 ARR(%) 32.1 47 7.6 NNT 2.8 2 13

*In the ANCHOR trial,8 patients with wet age-related macular degeneration were randomized to receive either Ranibizumab or verteporn. The outcome measure reported here is loss of 15 or more letters from baseline visual acuity at 12 months. wIn the Herpetic Eye Disease I (HEDS 1) study9- Stromal Keratitis, Not on Steroids Trial, patients with active HSV stromal keratitis diagnosed clinically were randomized to receive either topical antivirals or topical antivirals plus topical steroids for a 10-week period in tapered doses. The outcome measure reported here is treatment failure before the completion of a 10-week course of trial medication. zData reported here from the Age-Related Eye Disease Study (AREDS)10 show results from patients with categories 3 and 4 dry age-related macular degeneration (advanced dry disease) who were randomized to either antioxidants plus zinc or placebo. These show the probability of development of advanced age-related macular degeneration over a 5-year period.

after laser treatment. In 2006, Gillies and others11 conducted a well-designed RCT and showed that patients with refractory diabetic macular edema receiving 4-mg intravitreal triamcinolone acetonide (IVTA) (average 2.6 treatments) were signicantly more likely than patients receiving sham to gain 5 or more letters of best-corrected visual acuity after 2 years. However, IOP complications of the steroid were statistically and clinically signicant. At 2 years, more patients in the IVTA group required initiation of glaucoma medication (44% vs. 3%), and more patients in the IVTA group required glaucoma ltration surgery (6% vs. 0%) when compared with patients receiving sham. Therefore, the attributable risk for requiring initiation of glaucoma medication was 41% (44% to 3%) and the NNH for this adverse event was 2.4 (1/0.41). In addition, the attributable risk for requiring glaucoma ltration surgery is 6% (6% to 0%) and the NNH is 17 (1/0.06). Consequently, the complication of requiring initiation of glaucoma medication in 1 patient occurred when only 2.4 patients were exposed to IVTA treatment. Analogously, glaucoma ltration surgery was required in 1 out of every 17 patients exposed to IVTA therapy.

application in practice usually requires reproducible results from a good quality RCT or meta-analysis showing a statistically signicant treatment effect measured with a clinically meaningful primary outcome measure. As most new treatment modalities are associated with cost and potential adverse events, the treatment should also be considered clinically signicant. In other words, the size of the treatment effect should be large enough to justify its use in clinical practice. A relative risk ratio with associated P value is often quoted in the literature. However, to gain a full understanding of the size of a treatment effect and thus its clinical implications, the ARR and NNT are important concepts. The NNT is especially easy to conceptualize and quanties the number of patients that need therapy to prevent 1 harmful outcome. These statistics may not be reported outright in a published manuscript so it is worthwhile to be familiar with creating a simple 2 2 contingency table and performing the calculations by hand.

References
1. Rosner B. Fundamentals of Biostatistics. 6th ed. Toronto: Thomson; 2006. 2. Kertes PJ, Johnson TM. Evidence-Based Eye Care. Philadelphia: Lippincott Williams & Wilkins; 2007.

75

Conclusions
In evidence-based medicine, the consideration of a novel treatment modality for

3. Oxman AD, Sackett DL, Guyatt GH. Users guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA. 1993;270:20932095. 4. Sackett DL, Brian Haynes R, Tugwell P. Clinical Epidemiology a Basic Science for Clinical Medicine. Toronto: Little, Brown, and Company; 1988. 5. Oleckno W. Essential Epidemiology Principles and Applications. Prospect Heights, IL: Waveland Press; 2002. 6. Higginbotham EJ, Gordon MO, Beiser JA, et al. The Ocular Hypertension Treatment Study: topical medication delays or prevents primary open-angle glaucoma in African American individuals. Arch Ophthalmol. 2004;122:813820. 7. Kass MA, Heuer DK, Higginbotham EJ, et al. The Ocular Hypertension Treatment Study: a randomized trial determines that topical ocular hypotensive medication delays or prevents the onset of primary open-angle glaucoma. Arch Ophthalmol. 2002;120:701713; discussion 829830.

8. Brown DM, Kaiser PK, Michels M, et al. Ranibizumab Versus verteporn for neovascular agerelated macular degeneration. N Engl J Med. 2006;355:14321444. 9. Wilhelmus KR, Gee L, Hauck WW, et al. Herpetic Eye Disease Study. A controlled trial of topical corticosteroids for herpes simplex stromal keratitis. Ophthalmology. 1994;101: 18831895; discussion 18951896. 10. Age-Related Eye Disease Study Research Group. A randomized, placebo-controlled, clinical trial of high-dose supplementation with vitamins C and E, beta carotene, and zinc for age-related macular degeneration and vision loss: AREDS report no. 8. Arch Ophthalmol. 2001;119: 14171436. 11. Gillies MC, Sutter FK, Simpson JM, et al. Intravitreal triamcinolone for refractory diabetic macular edema: two-year results of a double-masked, placebo-controlled, randomized clinical trial. Opthalmology. 2006;113: 15331538.

76