PH 401: Meta-analysis

Eunice Pyon, PharmDeunice.pyon@liu.edu

(718) 488-1246, HS 506

Meta-analysis Quantitative systematic review Combines data from previously

conducted clinical trials (and epidemiologic research) and performs statistical analyses on pooled results

NOTE: different from a review article

Meta-analysis

Useful when:1. definitive clinical trials are impossible,

unethical or impractical2. randomized trials have been performed but

results are conflicting3. results from definitive trials are being

awaited4. new questions not posed at the beginning of

the trial need to answered5. sample sizes are too small

Meta-analysis

Purposes include:1. to increase statistical power for primary

endpoint and or subgroups2. to resolve uncertainty when reports disagree3. to improve estimates of size of effect4. to answer new questions not posed at the

start of the trials5. to bring about improvements in quality of

primary research

Meta-analysis: SSRIs Whittington CJ, et al. Selective

serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 2004;363:1341-1345.

Meta-analysis: Cox-2

Association Between Valdecoxib and Cardiovascular Events*

Number

Cardiovascular Events

Study

Valdecoxib

Placebo

Valdecoxib

Placebo

RR

95% CI

Ott et al3 311 151 14 2 3.40 0.82–13.98

2nd CABG4 1088 548 17 3 2.85 0.81–10.02

Meta-analytic RR5 ... ... ... ... 3.08 1.20–7.87

"Valdecoxib" indicates valdecoxib alone or in combination with parecoxib. Meta-analytic P=0.019; P heterogeneity=0.86.

*Cardiovascular events include coronary and cerebrovascular events.

Furberg CD, et al. Parecoxib, valdecoxib, and cardiovascular risk. Circulation 2005;111:249.

Meta-analysis: the process

1. Problem formulation2. Data collection3. Evaluation of the collected data4. Analysis and interpretation5. Presentation of Results

Problem formulation Clearly define the clinical question

specify variables evaluate relationship between

variables (cause and effect)

Data collection Describe details of literature search

databases published vs. unpublished additional sources (i.e.., reference lists,

meetings) Describe inclusion/exclusion criteria

study design participants treatment outcome measures

Evaluation of dataEven before the “data” Author Funding Relevant information

Important part of data evaluation.Different ways to incorporate into meta-

analysis: exclusion, weighting, stratifying

Evaluation of data Raw data, individual patient data

preferred difficult to obtain

Summary data more commonly utilized

Evaluation of data Homogeneity vs. heterogeneity

L’abbe plot Cochran-Q

Publication Bias Funnel plot

Funnel Plot

Meta-analysis continued

Remember:

Meta-analysis is an observational study of evidence. It is retrospective.

Evaluation of data Scrutinize validity of trials

randomization techniques sample size compliance blinding intention to treat vs. per protocol

Primary studies may be weighted to reflect quality of research design. Weighting of data is controversial.

Investigators should be blinded to: authors, institutions, journals, funding, acknowledgements.

Analysis and interpretation Appropriate statistical analyses

standardized outcome measure Continuous (i.e., blood pressure): differences,

standard deviations Binary (i.e., dead or alive): odds ratio, relative risk

overall effect; combining data fixed effects model--assumes same effect across

studies random effects model--assumes different

underlying effect for all studies and others…

Analysis and interpretation Odds Ratio

Sample Group Disease No Disease

Treatment or exposure a b

Control or no exposure c d

Total a+c b+d

controlsin exposure of Odds

casesin exposure of OddsOR

bc

ad

b/d

a/cOR

d

b

d)d/(b

d)b/(bcontrolsin exposure of Odds

c

a

c)c/(a

c)a/(acasesin exposure of Odds

Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men.

Lung cancer cases Controls

Smokers 647 622

Non-smokers  2 27

OR=?

Analysis and Interpretation: Odds Ratio

Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii 739-748). Results for men.

Lung cancer cases Controls

Smokers 647 622

Non-smokers  2 27

Odds ratio = (647x27) / (2x622) = 14.04Lung cancer cases 14 x more likely to be smokers.

Analysis and Interpretation: Odds Ratio

Analysis and interpretation Relative Risk

d)c/(c

b)a/(a

group controlin event ofy Probabilit

groupnt in treatmeevent ofy ProbabilitRR

Sample Group Disease No Disease

Total

Treatment or exposure

a b a+b

Control or no exposure

c D c+d

Total a+c b+d

Analysis and interpretation Sensitivity analysis

Overall effect calculated by different methods (fixed vs. random)

Reanalysis with exclusion of poor-quality studies

Reanalysis with exclusion of small studies

Reanalysis of exclusion of studies with short duration of follow-up

Presentation of results Often graphically displayed with

confidence intervals Type I and II error should be

discussed Robustness of findings/sensitivity

should be discussed

Strengths Can summarize from available

studies the effects of interventions across many patients

Can reveal research designs as moderators of study results

Can reduce false negative results Can clarify heterogeneity between

study results

Strengths Can assist in accurate calculation

of sample size needed in future studies

Can suggest promising research questions for future study

Can allow more objective assessment of evidence and thereby reduce disagreement

Weaknesses Can pass along inflated estimates

of size effects based previously reported results

Cannot overcome subjectivity in choice of outcomes and their weighting in analysis

Can be compromised by publication bias

Weaknesses Arithmetic nature of meta-analysis

can produce false impression of certainty in an inherently uncertain process with many subjective elements

Cochrane Collaboration The Cochrane Collaboration is an

international not-for-profit organization, providing information about the effects of health care

Source of qualitative and quantitative systematic reviews with good methodological rigor

www.cochrane.org

Conclusions Interpret with caution

remembering that conclusions depend on the quality of the studies included

Findings of subsequent randomized controlled trials may differ

References Malone PM et al. Drug information: a

guide for pharmacists. McGraw-Hill. New York. 2nd edition. 2001.

Noble Jr JH. Meta-analysis: methods, strengths, weaknesses, and political uses. J Lab Clin Med 2006;147:7-20.