AHRQ annual meetingAHRQ annual meetingSeptember 10, 2008September 10, 2008

Stephanie Chang MD, MPHStephanie Chang MD, MPHCenter for Outcomes and EvidenceCenter for Outcomes and Evidence

Conducting a methodologically Conducting a methodologically sound systematic review with sound systematic review with

limited resources: An introductionlimited resources: An introduction

ObjectivesObjectives

Understand breadth of methods issues in Understand breadth of methods issues in conducting systematic reviews conducting systematic reviews

Identify available resources for methods Identify available resources for methods guidanceguidance

Identify available resources for conducting a Identify available resources for conducting a reviewreview

Conducting systematic reviewsConducting systematic reviews

Elements of systematic reviewElements of systematic review

Clarify critical questionsClarify critical questions Specify appropriate sources of information to address Specify appropriate sources of information to address

questionsquestions Evaluate quality of individual studiesEvaluate quality of individual studies

– Internal and external validityInternal and external validity Summarize evidence for each questionSummarize evidence for each question

– Qualitative synthesis, Meta-analysisQualitative synthesis, Meta-analysis– (Decision analysis, Cost-effectiveness)(Decision analysis, Cost-effectiveness)

Methodology of systematic Methodology of systematic reviewsreviews

Relevant, but independentRelevant, but independent– Engaging partnersEngaging partners for Key Questions for Key Questions – Engage partners for translationEngage partners for translation– Technical expertTechnical expert panel panel

Comprehensive, rigorous, unbiasedComprehensive, rigorous, unbiased– Complete literature reviewComplete literature review– Assessment of study qualityAssessment of study quality– A priori definitionsA priori definitions– Peer reviewPeer review

TransparentTransparent– Reporting of methods usedReporting of methods used– Methods GuideMethods Guide

AHRQ Evidence-based Practice AHRQ Evidence-based Practice CentersCenters

BC/BS TECBC/BS TEC U. ConnU. Conn Duke U.Duke U. ECRIECRI Johns Hopkins U.Johns Hopkins U. McMaster U.McMaster U. U. MinnesotaU. Minnesota

Oregon HSUOregon HSU U. So California. - RANDU. So California. - RAND RTI – UNCRTI – UNC TuftsTufts UCSF - Stanford U.UCSF - Stanford U. U. AlbertaU. Alberta U. OttawaU. Ottawa

Evidence-based Practice Centers Evidence-based Practice Centers conduct systematic reviewsconduct systematic reviews

Methods Guide ResourcesMethods Guide Resources

Methods Guide for Conducting Comparative Methods Guide for Conducting Comparative Effectiveness ReviewsEffectiveness Reviews– Topic DevelopmentTopic Development– Selecting EvidenceSelecting Evidence– Finding EvidenceFinding Evidence– Assessing Quality of individual studiesAssessing Quality of individual studies– Assessing ApplicabilityAssessing Applicability– Assessing HarmsAssessing Harms– Quantitative SynthesisQuantitative Synthesis– Rating a body of evidenceRating a body of evidence• www.effectivehealthcare.ahrq.gov/www.effectivehealthcare.ahrq.gov/• http://effectivehealthcare.ahrq.gov/repFiles/http://effectivehealthcare.ahrq.gov/repFiles/

2007_10DraftMethodsGuide.pdf2007_10DraftMethodsGuide.pdf

Cochrane Handbook and GlossaryCochrane Handbook and Glossary– http://www.cochrane.org/resources/handbook/http://www.cochrane.org/resources/handbook/

Software toolsSoftware tools Data ManagementData Management Evidence TablesEvidence Tables Meta-analysisMeta-analysis

RevManRevMan– http://www.cc-ims.net/RevManhttp://www.cc-ims.net/RevMan

GradeProGradePro– http://www.cc-ims.net/gradeprohttp://www.cc-ims.net/gradepro

Comprehensive Meta-analysis ($)Comprehensive Meta-analysis ($)– http://www.meta-analysis.com/http://www.meta-analysis.com/

Statistical softwareStatistical software– STATA, SAS, SPSS ($)STATA, SAS, SPSS ($)

TrialStat! ($)TrialStat! ($)– http://www.trialstat.com/http://www.trialstat.com/

Questions? Questions?

Stephanie ChangStephanie ChangStephanie.chang@Stephanie.chang@ahrq.hhs.govahrq.hhs.gov