META-ANALYSIS AND SYSTEMATIC

REVIEW
Gianluigi Savarese, MD, FESC, ACC FIT

Department of Advanced Biomedical Sciences, Federico II University, Naples,

Italy
Department of Medicine, Solna, Karolinska Institutet, Stockholm, Sweden

What is a Systematic
Review?
A review that is conducted according
to clearly stated, scientific research
methods, and is designed to minimize
biases
and
errors
inherent
to
traditional, narrative reviews.

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD, Clinical Perspective: A Practical Guide to Meta-Analysis. The
Journal of Hand Surgery. Vol. 31A No.10 December 2006. p.1671

What is the significance of

Systematic Reviews?

The large amount of medical literature requires

clinicians and researchers alike to rely on systematic
reviews in order to make an informed decision.

Systematic Reviews minimize bias. A systematic review

is a more scientific method of summarizing literature
because specific protocols are used to determine which
studies will be included in the review.

Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p.1839

Why are Systematic Reviews

Necessary?
The volume of published material makes it impractical for an
individual clinician to remain up to date on a variety of
common conditions. This is further complicated when
individual studies report conflicting conclusions, a problem
that is prevalent when small patient samples and
retrospective designs are used.

Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p.1839

Characteristics of Systematic
Reviews
Two possible approaches:

or

qualitative

synthesis

statistical synthesis of data (metaanalysis) if appropriate and possible

Hypothesis
A systematic review should be based
on principles of hypothesis testing, and
the hypotheses must be conceived a
priori.

Four steps
Identify your studies
Determine eligibility of studies
apriori to avoid bias
Inclusion: which ones to keep
Exclusion: which ones to throw out

Abstract Data from the studies

Analyze data in the studies
statistically

Literature Search
A comprehensive and reproducible
literature search is the foundation of a
systematic review.

Literature Search
Be methodical: plan first
List of popular databases to search

Pubmed/Medline
Embase
Cochrane Review
ISI Web of Science
SCOPUS

Database bias!!!

Other strategies you may adopt

Trial registries (clinicaltrials.gov)

Abstracts from meetings
Hand search (go to the library...)
Grey
Personal references
litterature
References from published reviews/meta-analysis/trials
Contact experts
Web, eg. Google (http://scholar.google.com)

Literature Search Risk of

Bias
English-language bias -

occurs when reviewers

exclude papers published in languages other than English

Citation bias -

occurs when studies with significant or

positive results are referenced in other publications,
compared with studies with inconclusive or negative
findings

Publication Bias -

selective publication of articles

that show positive treatment of effects and statistical
significance.

Data Collection
The list of data to be extracted should be decided
a priori.
A data extraction form should be used so that the
same data are extracted from each study and
missing data are clearly apparent.
To be sure that data extraction is accurate and
reproducible, it should be performed by at least
two independent readers.
Disagreement between readers could be solved by
agreements or by a third reviewer
Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p.1839

Data Collection
Collected data includes:
Study characteristics (year and journal of
publication, number of patients in each arm,
treatments performed, duration of follow-up)
Sample demographics (age, % males or
females)
Sample characteristics (traditional CV risk
factors - % hypertensive pts, % diabetic pts, %
dyslipidemic pts, % smokers concomitant
treatments, comorbidities, etc)
Outcome data (all-cause death, CV death, MI,
stroke, etc)

Quality Assessment
The validity of a systematic review
ultimately depends on the scientific
method of the retrieved studies and the
reporting of data.

Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p.1839

GRADE
Grading
of
Recommendations
Assessment, Development
and Evaluation

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schnemann HJ; GRADE Working Group. What is "quality of evidence" and
why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schnemann HJ; GRADE Working Group. What is "quality of evidence" and
why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.

Data Synthesis
Data
could
be
summarized
quantitatively if study designs are not
too different in:
outcome
definition
(composite
outcome?);
population sizes
population characteristics
interventions
HETEROGENEITY

Data Synthesis
Once the data have been extracted
and their quality and validity assessed,
the outcomes of individual studies
within a systematic review may be
pooled and presented as summary
outcome or effect
META-ANALYSIS

What is meta analysis?

Quantitative approach for systematically
combining results of previous research to
arrive at conclusions about the body of
research.

What does it mean?

Quantitative : numbers
Systematic : methodical
combining: putting together
previous research: what's
already done
conclusions: new knowledge

Meta-analysis: Statistical
Models
There are 2 statistical models used in a
meta-analysis:
Fixed effects:
1.Effect of treatment is the same for every study;
2.Low heterogeneity
Random effects:
1.True effect estimate for each study varies;
2.High heterogeneity
3.Provide larger CI

Heterogeneity
Clinical heterogeneity:
variability in the
participants, interventions and outcomes studied
+
Methodological heterogeneity: variability in
study design

Statistical heterogeneity: Variability in the

intervention effects being evaluated in the
different studies. It is a consequence of clinical or
methodological diversity, or both, among the
studies.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March
2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Heterogeneity assessment
If confidence intervals for the results of individual studies
(generally depicted graphically using horizontal lines) have poor
overlap, this generally indicates the presence of statistical
heterogeneity.
Cochrane Q statistic: It is calculated as the weighted sum of
squared differences between individual study effects and the
pooled effect across studies, with the weights being those used
in the pooling method. A p value decided apriori defines the
presence of significant heterogeneity.
I2 statistic: It describes the percentage of variation across studies
that is due to heterogeneity rather than chance.
0% to 40%: heterogeneity might not be important;
30% to 60%: may represent moderate heterogeneity;
50% to 90%: may represent substantial heterogeneity;
75% to 100%: considerable heterogeneity.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March
2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Strategies for addressing

heterogeneity
Check again that the data are correct
Do not do a meta-analysis
Explore heterogeneity (subgroup analysis,
meta-regression)
Ignore heterogeneity (there is no an
intervention effect but a distribution of
intervention effects)
Perform a random-effects meta-analysis
(when heterogeneity cannot be explained)
Change the effect measure (different scales
in different studies)
Exclude studies (outlying studies)
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March
2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Sensitivity analysis
One study removed meta-analysis
Meta-regression analysis

Publication Bias
Publication bias arises when trials with
statistically significant results are more
likely to be published and cited, and
are preferentially published in English
language journals and those indexed in
Medline
Jni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials:
empirical study. International Journal of Epidemiology 2001;31:115-123.

Publication Bias
A funnel plot is a simple scatter plot of the
intervention effect estimates (OR, logOR) from
individual studies against some measure of each
studys size or precision (standard error,
1/standard error, sample size, 1/sample size,
log(sample size), log(1/sample size), MantelHaenszel weight).
The best choice of x axis for detecting the small
sample effect is the log odds ratio. This is
because the scale is not constrained and
because the plot will be the same shape whether
the outcome is defined as occurrence or nonoccurrence of event.
Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical
Epidemiology 2001;54:1046-1055.

Symmetrical plot in the absence of

bias (open
circles
indicate
smaller
studies
showing no beneficial effects)

Asymmetrical plot in the presence of

publication bias (smaller studies
showing no beneficial effects are
missing)

Asymmetrical plot in the presence of

bias due to low methodological
quality of smaller studies (open
circles indicate small studies of
inadequate quality whose results are
biased towards larger beneficial
Jonathan A et al. The Stata Journal
effects)
2004; 4:127

Publication Bias

Ntot is the total sample size, NE and NC are the sizes of the experimental
and control intervention groups, S is the total number of events across both
groups and F = Ntot S. Note that only the first three of these tests (Begg
1994, Egger 1997a, Tang 2000) can be used for continuous outcomes.

Protocols
The purpose of PRISMA (Preferred
Reporting
Items
for
Systematic
Reviews and Meta-Analyses) guidelines
is to provide proper procedures for
conducting a meta-analysis and to
standardize the methods of reporting a
meta-analysis.

Savarese G et al J Am Coll Cardiol

2013;61:131

Background
Angiotensin-converting enzyme inhibitors (ACE-Is)
are recommended for reduction of cardiovascular
(CV) events in patients at high CV risk without
heart failure (HF).
In contrast, CV effects of angiotensin receptor
blockers (ARBs) on major clinical outcomes in
patients without HF are less certain as major
clinical trials comparing ARBs vs placebo reported
conflicting results.

Savarese G et al J Am Coll Cardiol

2013;61:131

Methods Inclusion Criteria

Report of at least one clinical outcome (all-cause

death, CV death, myocardial infarction, stroke,
new onset heart failure, new onset diabetes
mellitus).
Randomized, placebo-controlled trials using ACEIs or ARBs as treatments.

Savarese G et al J Am Coll Cardiol

2013;61:131

Methods Statistical
methods
Meta-analysis was performed to assess
influence of treatments on outcomes.

the

Statistical homogeneity was assessed using Q

statistic and further quantified with the I2 statistic.
Meta-regression was performed to test the
influence of potential effect modifiers on results.
Publication bias was assessed using linear
regression test by Egger and Macaskills modified
test.
Savarese G et al J Am Coll Cardiol
2013;61:131

Results Search Strategy

Savarese G et al J Am Coll Cardiol

2013;61:131

Results Population
characteristics

Results Population
characteristics

Results Population
characteristics

Results Composite
Outcome
ARBs significantly reduced
the risk of the composite
outcome
by
7.0%
compared
to
placebo
(p=0.012).

ACE-Is significantly reduced

the risk of the composite
outcome
by
14.9%
compared
to
placebo
(p=0.001).

Savarese G et al J Am Coll Cardiol

2013;61:131

Results CV Death

ARBs did not reduce

the risk of CV death
(p=0.768)

10% reduction of CV
death did not achieve
statistical significance
in
ACE-Is
trials
(p=0.087).

Savarese G et al J Am Coll Cardiol

2013;61:131

Results Myocardial
infarction
9.5% reduction of MI risk
did not achieve statistical
significance in ARBs trials
(p=0.086).

ACE-Is
significantly
reduced the risk of MI by
17.7% (p<0.001)

Savarese G et al J Am Coll Cardiol

2013;61:131

Results Stroke

ARBs
significantly
reduced the risk of
stroke
by
9.1%
(p=0.011).

ACE-Is
significantly
reduced the risk of
stroke
by
19.6%
(p=0.004)

Savarese G et al J Am Coll Cardiol

2013;61:131

Results All Cause Death

No significant effect was
found on the risk of allcause death in ARBs trials
(p=0.866).

ACE-Is reduced the risk of

all-cause death by 8.3%
(p=0.008).

Savarese G et al J Am Coll Cardiol

2013;61:131

Results New onset HF

No significant effect
was found on the
risk of new onset
HF in ARBs trials
(p=0.866).

ACE-Is reduced the

risk of new onset
HF
by
20.5%
(p=0.001).

Savarese G et al J Am Coll Cardiol

2013;61:131

Results New onset DM

ARBs
significantly
reduced
the
risk of new
onset DM by
10.6%
(p<0.01).
ACE-Is reduced
the risk of new
onset diabetes
by
13.7%
(p=0.012)

Savarese G et al J Am Coll Cardiol

2013;61:131

Results Subgroup analysis

Savarese G et al J Am Coll Cardiol

2013;61:131

Results Meta-regression
analysis

Conclusions
In
comparison
to
placebo,
ACE-Is
substantially reduce the composite of CV
death/MI/stroke as well as all-cause death,
new onset HF and new onset DM in highrisk patients without HF, mostly with
coronary or other vascular diseases.
ARBs, in high-risk patients mostly with DM
or impaired glucose tolerance, without HF,
reduce the composite outcome and new
onset DM, but do not appear to reduce
rates of all-cause death or new onset HF.

Savarese G et al J Am Coll Cardiol

2013;61:131

Types of Metaanalysis/Terminology
Systematic Review

Meta-analysis Extract data from published reports

(Overview)
(aggregated data meta-analysis)

Frequentist Approach
Bayesian Approach
Network
Collect individual patient data (IPD)

Frequentist approach
Classical methods are, usually based on algorithms using
explicit formulas.
main
assumpti
onsof the
model

results of
studies
(usually
RCTs)

Transformati
ons of input
data

Results of Meta-analysis

Bayesian Meta-Analysis
Including extra (prior)
information
main
assumpti
onsof the
model

establishing
prior
distributions
basing on:

Extra data e.g.

results of nonrandomized
trials, historical
observations,
Settingetc.
the level of
conviction to this
data
MCMC
simulations

Results of Meta-analysis

results of
randomized
studies

Bayesian Meta-Analysis
Assessing clinical significance
main
assumpti
onsof the
model

noninformative
prior
distributions

results of
studies

establishing the
level of clinical
significant result
(e.g. RR > 1.2)
MCMC
simulations
Results of Meta-analysis

Answering the question: How

probable is that the result is
clinically significant?

Possible to
obtain due to
knowledge of
whole
distribution

Frequentist vs Bayesian
approach

Bayesian approach

Frequentialist
methods

philosoph
y

First: assumptions and

construction
then: inputing results of studies

Construction
based on the
results of studies

flexibility

YES

NO

computati
on

Makov Chain Monte Carlo

simulations

formulas

software

specialistic, e.g. WinBUGS

no special
requirements

Network Meta-Analysis
(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)

Combine direct + indirect estimates of multiple treatment effects

Internally consistent set of estimates that respects randomization
Estimate effect of each intervention relative to every other
whether or not there is direct comparison in studies
Calculate probability that each treatment is most effective
Compared to conventional pair-wise meta-analysis:
Greater precision in summary estimates
Ranking of treatments according to effectiveness
55

Indirect Comparisons of Multiple

Treatments Network MetaAnalysis
Trial
1

Want to compare A vs. B

Direct evidence from trials 1, 2 and 7
Indirect evidence from trials 3, 4, 5, 6 and 7

Combining all A arms and

comparing with all B arms destroys
randomization
Use indirect evidence of A vs. C and
B vs. C comparisons as additional
evidence to preserve randomization
and within-study comparison

Network Meta-Analysis
(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)
paroxetine

reboxetine

duloxetine

mirtazapine

escitalopram

fluvoxamine

milnacipran

citalopram

sertraline

venlafaxine

bupropion

fluoxetine

milnacipran
sertraline

paroxetine
?

duloxetine

bupropion

escitalopram

fluvoxamine

milnacipran

19 meta-analyses of pairwise comparisons published

What is an individual patients

data Meta-analysis?
Involves the central collection, checking
and analysis of updated individual
patient data
Include all properly randomised trials,
published and unpublished
Include all patients in an intention-totreat analysis

Benefits of IPD

Carry out time-to-event analyses

Only practical way to do subgroup analyses
More flexible analysis of outcomes
Carry out detailed data checking
Ensure quality of randomisation and follow up
Ensure appropriateness of analysis
Update follow up information

Other Benefits

More complete identification of trials

Better compliance in providing missing data
More balanced interpretation of results
Wider endorsement and dissemination of results
Better clarification of further research
Collaboration on further research

THANKS FOR THE ATTENTION!!!

Sorrento, Naples, Italy.