META-ANALYSIS AND SYSTEMATIC REVIEW Gianluigi Savarese, MD, FESC, ACC FIT Department of Advanced...
-
Upload
elwin-whitehead -
Category
Documents
-
view
216 -
download
0
Transcript of META-ANALYSIS AND SYSTEMATIC REVIEW Gianluigi Savarese, MD, FESC, ACC FIT Department of Advanced...
META-ANALYSIS AND SYSTEMATIC REVIEW
Gianluigi Savarese, MD, FESC, ACC FIT
Department of Advanced Biomedical Sciences, Federico II University, Naples, ItalyDepartment of Medicine, Solna, Karolinska Institutet, Stockholm, Sweden
What is a Systematic Review?
“A review that is conducted according to clearly stated, scientific research methods, and is designed to minimize biases and errors inherent to traditional, narrative reviews.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD, “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol. 31A No.10 December 2006. p.1671
What is the significance of Systematic Reviews?
• The large amount of medical literature requires clinicians and researchers alike to rely on systematic reviews in order to make an informed decision.
• Systematic Reviews minimize bias. “A systematic review is a more scientific method of summarizing literature because specific protocols are used to determine which studies will be included in the review.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
Why are Systematic Reviews Necessary?
“The volume of published material makes it impractical for an individual clinician to remain up to date on a variety of common conditions. This is further complicated when individual studies report conflicting conclusions, a problem that is prevalent when small patient samples and retrospective designs are used.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
Characteristics of Systematic Reviews
• Two possible approaches:
– or qualitative synthesis
– statistical synthesis of data (meta-analysis) if appropriate and possible
Hypothesis
A systematic review should be based on principles of hypothesis testing, and the hypotheses must be conceived a priori.
• Identify your studies• Determine eligibility of studies apriori to
avoid bias– Inclusion: which ones to keep– Exclusion: which ones to throw out
• Abstract Data from the studies• Analyze data in the studies statistically
Four steps
Literature Search
A comprehensive and reproducible literature search is the foundation of a systematic review.
• Be methodical: plan first• List of popular databases to search
– Pubmed/Medline– Embase– Cochrane Review– ISI Web of Science– SCOPUS
• Other strategies you may adopt– Trial registries (clinicaltrials.gov)– Abstracts from meetings– Hand search (go to the library...)– Personal references– References from published reviews/meta-analysis/trials– Contact experts – Web, eg. Google (http://scholar.google.com)
Literature Search
Database bias!!!
Grey litterature
Literature Search – Risk of Bias
• English-language bias - occurs when reviewers exclude papers published in languages other than English
• Citation bias - occurs when studies with significant or positive results are referenced in other publications, compared with studies with inconclusive or negative findings
• Publication Bias - selective publication of articles that show positive treatment of effects and statistical significance.
Data Collection• The list of data to be extracted should be decided a priori.
• A data extraction form should be used so that the same data are extracted from each study and missing data are clearly apparent.
• To be sure that data extraction is accurate and reproducible, it should be performed by at least two independent readers.
• Disagreement between readers could be solved by agreements or by a third reviewer
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
Data Collection
Collected data includes:– Study characteristics (year and journal of publication,
number of patients in each arm, treatments performed, duration of follow-up)
– Sample demographics (age, % males or females)
– Sample characteristics (traditional CV risk factors - % hypertensive pts, % diabetic pts, % dyslipidemic pts, % smokers – concomitant treatments, comorbidities, etc)
– Outcome data (all-cause death, CV death, MI, stroke, etc)
Quality Assessment
The validity of a systematic review ultimately depends on the scientific method of the retrieved studies and the reporting of data.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839
GRADE
Grading of Recommendations Assessment, Development and Evaluation
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.
Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.
Data Synthesis
Data could be summarized quantitatively if study designs are not too different in: •outcome definition (composite outcome?); •population sizes•population characteristics•interventions
HETEROGENEITY
Data Synthesis
Once the data have been extracted and their quality and validity assessed, the outcomes of individual studies within a systematic review may be pooled and presented as summary outcome or effect
META-ANALYSIS
What is meta analysis?
Quantitative approach for systematically combining results of previous research to
arrive at conclusions about the body of research.
What does it mean?
• Quantitative : numbers• Systematic : methodical• combining: putting together• previous research: what's already done• conclusions: new knowledge
Meta-analysis: Statistical Models
• There are 2 statistical models used in a meta-analysis: – Fixed effects:1.Effect of treatment is the same for every study;2.Low heterogeneity
– Random effects:1.True effect estimate for each study varies;2.High heterogeneity3.Provide larger CI
Heterogeneity• Clinical heterogeneity: variability in the participants,
interventions and outcomes studied+
• Methodological heterogeneity: variability in study design
• Statistical heterogeneity: Variability in the intervention effects being evaluated in the different studies. It is a consequence of clinical or methodological diversity, or both, among the studies.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
Heterogeneity assessment• If confidence intervals for the results of individual studies (generally
depicted graphically using horizontal lines) have poor overlap, this generally indicates the presence of statistical heterogeneity.
• Cochrane Q statistic: It is calculated as the weighted sum of squared differences between individual study effects and the pooled effect across studies, with the weights being those used in the pooling method. A p value decided apriori defines the presence of significant heterogeneity.
• I2 statistic: It describes the percentage of variation across studies that is due to heterogeneity rather than chance.
• 0% to 40%: heterogeneity might not be important;• 30% to 60%: may represent moderate heterogeneity;• 50% to 90%: may represent substantial heterogeneity;• 75% to 100%: considerable heterogeneity.
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
Strategies for addressing heterogeneity
• Check again that the data are correct• Do not do a meta-analysis• Explore heterogeneity (subgroup analysis, meta-
regression)• Ignore heterogeneity (there is no an intervention
effect but a distribution of intervention effects)• Perform a random-effects meta-analysis (when
heterogeneity cannot be explained)• Change the effect measure (different scales in different
studies)• Exclude studies (outlying studies)
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
Sensitivity analysis
• One study removed meta-analysis• Meta-regression analysis
Publication Bias
Publication bias arises when trials with statistically significant results are more likely to be published and cited, and are preferentially published in English language journals and those indexed in Medline
Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. International Journal of Epidemiology 2001;31:115-123.
Publication Bias
• A funnel plot is a simple scatter plot of the intervention effect estimates (OR, logOR) from individual studies against some measure of each study’s size or precision (standard error, 1/standard error, sample size, 1/sample size, log(sample size), log(1/sample size), Mantel-Haenszel weight).
• The best choice of x axis for detecting the small sample effect is the log odds ratio. This is because the scale is not constrained and because the plot will be the same shape whether the outcome is defined as occurrence or non-occurrence of event.
Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology 2001;54:1046-1055.
Symmetrical plot in the absence of bias (opencircles indicate smaller studies showing no beneficial effects)
Asymmetrical plot in the presence of publication bias (smaller studies showing no beneficial effects are missing)
Asymmetrical plot in the presence of bias due to low methodological quality of smaller studies (open circles indicate small studies of inadequate quality whose results are biased towards larger beneficial effects)
Jonathan A et al. The Stata Journal 2004; 4:127
Publication Bias
Ntot is the total sample size, NE and NC are the sizes of the experimental and control intervention groups, S is the total number of events across both groups and F = Ntot – S. Note that only the first three of these tests (Begg 1994, Egger 1997a, Tang 2000) can be used for continuous outcomes.
Protocols
The purpose of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines is to provide proper procedures for conducting a meta-analysis and to standardize the methods of reporting a meta-analysis.
Savarese G et al J Am Coll Cardiol 2013;61:131
• Angiotensin-converting enzyme inhibitors (ACE-Is) are recommended for reduction of cardiovascular (CV) events in patients at high CV risk without heart failure (HF).
• In contrast, CV effects of angiotensin receptor blockers (ARBs) on major clinical outcomes in patients without HF are less certain as major clinical trials comparing ARBs vs placebo reported conflicting results.
Background
Savarese G et al J Am Coll Cardiol 2013;61:131
Methods – Inclusion Criteria
• Report of at least one clinical outcome (all-cause death, CV death, myocardial infarction, stroke, new onset heart failure, new onset diabetes mellitus).
• Randomized, placebo-controlled trials using ACE-Is or ARBs as treatments.
Savarese G et al J Am Coll Cardiol 2013;61:131
• Meta-analysis was performed to assess the influence of treatments on outcomes.
• Statistical homogeneity was assessed using Q statistic and further quantified with the I2 statistic.
• Meta-regression was performed to test the influence of potential effect modifiers on results.
• Publication bias was assessed using linear regression test by Egger and Macaskill’s modified test.
Methods – Statistical methods
Savarese G et al J Am Coll Cardiol 2013;61:131
Results – Search Strategy
Savarese G et al J Am Coll Cardiol 2013;61:131
Results – Population characteristics
Results – Population characteristics
Results – Population characteristics
ACE-Is significantly reduced the risk of the composite outcome by 14.9% compared to placebo (p=0.001).
ARBs significantly reduced the risk of the composite outcome by 7.0% compared to placebo (p=0.012).
Results – Composite Outcome
Savarese G et al J Am Coll Cardiol 2013;61:131
10% reduction of CV death did not achieve statistical significance in ACE-Is trials (p=0.087).
ARBs did not reduce the risk of CV death (p=0.768)
Results – CV Death
Savarese G et al J Am Coll Cardiol 2013;61:131
ACE-Is significantly reduced the risk of MI by 17.7% (p<0.001)
9.5% reduction of MI risk did not achieve statistical significance in ARBs trials (p=0.086).
Results – Myocardial infarction
Savarese G et al J Am Coll Cardiol 2013;61:131
ARBs significantly reduced the risk of stroke by 9.1% (p=0.011).
ACE-Is significantly reduced the risk of stroke by 19.6% (p=0.004)
Results – Stroke
Savarese G et al J Am Coll Cardiol 2013;61:131
ACE-Is reduced the risk of all-cause death by 8.3% (p=0.008).
No significant effect was found on the risk of all-cause death in ARBs trials (p=0.866).
Results – All Cause Death
Savarese G et al J Am Coll Cardiol 2013;61:131
No significant effect was found on the risk of new onset HF in ARBs trials (p=0.866).
ACE-Is reduced the risk of new onset HF by 20.5% (p=0.001).
Results – New onset HF
Savarese G et al J Am Coll Cardiol 2013;61:131
ACE-Is reduced the risk of new onset diabetes by 13.7% (p=0.012)
ARBs significantly reduced the risk of new onset DM by 10.6% (p<0.01).
Results – New onset DM
Savarese G et al J Am Coll Cardiol 2013;61:131
Results – Subgroup analysis
Savarese G et al J Am Coll Cardiol 2013;61:131
Results – Meta-regression analysis
• In comparison to placebo, ACE-Is substantially reduce the composite of CV death/MI/stroke as well as all-cause death, new onset HF and new onset DM in high-risk patients without HF, mostly with coronary or other vascular diseases.
• ARBs, in high-risk patients mostly with DM or impaired glucose tolerance, without HF, reduce the composite outcome and new onset DM, but do not appear to reduce rates of all-cause death or new onset HF.
Conclusions
Savarese G et al J Am Coll Cardiol 2013;61:131
Types of Meta-analysis/Terminology
Systematic Review
Meta-analysis(Overview)
Extract data from published reports(aggregated data meta-analysis)
Collect individual patient data (IPD)
Frequentist Approach
Bayesian Approach
Network
Frequentist approachClassical methods are, usually based on algorithms using explicit
formulas.
main assumptions
of the model
results of studies (usually
RCTs)
Transformations of input data
Results of Meta-analysis
Bayesian Meta-Analysis Including extra (prior) information
Extra data e.g. results of non-randomized
trials, historical observations, etc.
establishing prior
distributions basing on:
main assumptions
of the model
results of randomized
studies
Results of Meta-analysis
MCMC simulations
Setting the level of conviction to this data
!
Bayesian Meta-AnalysisAssessing clinical significance
Results of Meta-analysis
MCMC simulations
non-informative prior
distributionsresults of studies
Answering the question: How probable is that the result is clinically significant?
Possible to obtain due to knowledge of
whole distribution
establishing the level of clinical significant result
(e.g. RR > 1.2)
main assumptions
of the model
!
Frequentist vs Bayesian approach
Bayesian approach Frequentialist methods
philosophy First: assumptions and constructionthen: inputing results of studies
Construction based on the results of studies
flexibility YES NO
computation Makov Chain Monte Carlo simulations formulas
software specialistic, e.g. WinBUGS no special requirements
55
Network Meta-Analysis(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)
• Combine direct + indirect estimates of multiple treatment effects
• Internally consistent set of estimates that respects randomization
• Estimate effect of each intervention relative to every other whether or not there is direct comparison in studies
• Calculate probability that each treatment is most effective
• Compared to conventional pair-wise meta-analysis:
• Greater precision in summary estimates
• Ranking of treatments according to effectiveness
Indirect Comparisons of Multiple Treatments – Network Meta-Analysis
Trial
1 A B
2 A B
3 B C
4 B C
5 A C
6 A C
7 A B C
• Want to compare A vs. BDirect evidence from trials 1, 2 and 7Indirect evidence from trials 3, 4, 5, 6 and 7
• Combining all “A” arms and comparing with all “B” arms destroys randomization
• Use indirect evidence of A vs. C and B vs. C comparisons as additional evidence to preserve randomization and within-study comparison
paroxetine
sertraline
citalopram
fluoxetine
fluvoxamine
milnacipran
venlafaxine
reboxetine
bupropion
mirtazapineduloxetine
escitalopram
sertraline
milnacipran
bupropion
paroxetine
milnacipran
duloxetine
escitalopram
fluvoxamine
?
19 meta-analyses of pairwise comparisons published
Network Meta-Analysis(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)
What is an individual patients data Meta-analysis?
• Involves the central collection, checking and analysis of updated individual patient data
• Include all properly randomised trials, published and unpublished
• Include all patients in an intention-to-treat analysis
Benefits of IPD
• Carry out time-to-event analyses• Only practical way to do subgroup analyses• More flexible analysis of outcomes• Carry out detailed data checking• Ensure quality of randomisation and follow up• Ensure appropriateness of analysis• Update follow up information
Other Benefits
• More complete identification of trials• Better compliance in providing missing data• More balanced interpretation of results• Wider endorsement and dissemination of results• Better clarification of further research• Collaboration on further research
THANKS FOR THE ATTENTION!!!
Sorrento, Naples, Italy.