Overview of Meta-Analysis Approaches
Transcript of Overview of Meta-Analysis Approaches
![Page 1: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/1.jpg)
OverviewofMeta-AnalysisApproaches
ThomasE.NicholsUniversityofOxford
NeuroimagingMeta-AnalysisOHBMEducationalCourse
25June,2017slides&posters@http://warwick.ac.uk/tenichols/ohbm
![Page 2: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/2.jpg)
Overview
• Non-imagingmeta-analysis• Menuofmeta-analysismethods– ROI’s,IBMA,CBMA
• CBMAdetails– Kernel-basedmethods– What’sincommon– m/ALE,M/KDA– What’sdifferent
• Limitations&Thoughts
![Page 3: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/3.jpg)
Stagesof(non-imaging) Meta-Analysis
1. Definereview'sspecificobjectives.2. Specifyeligibilitycriteria.3. Identifyalleligiblestudies.4. Collectandvalidatedatarigorously.5. Displayeffectsforeachstudy,withmeasuresof
precision.6. Computeaverageeffect,randomeffectsstd err7. Checkforpublicationbias,conductsensitivity
analyses.
Jones,D.R.(1995).Meta-analysis:weighingtheevidence.StatisticsinMedicine,14(2),137–49.
![Page 4: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/4.jpg)
Methodsfor(non-imaging) Meta-Analysis(1)• P-value(orZ-value)combining– Fishers(≈average–logP)– Stouffers(≈averageZ)– Usedonlyasmethodoflastresort
• Basedonsignificance,noteffectsinrealunits• Differingn willinduceheterogeneity (Cummings,2004)
• Fixedeffectsmodel– Requireseffectestimatesandstandarderrors
• E.g.Meansurvival(days),andstandarderrorofmean– Givesweightedaverageofeffects
• Weightsbasedonper-studystandarderrors– Neglectsinter-studyvariation
Cummings(2004).Meta-analysisbasedonstandardizedeffectsisunreliable.ArchivesofPediatrics&AdolescentMedicine,158(6),595–7.
![Page 5: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/5.jpg)
Methodsfor(non-imaging) Meta-Analysis(2)• Randomeffectsmodel– Requireseffectestimatesandstandarderrors– Givesweightedaverageofeffect• Weightsbasedonper-studystandarderrorsandinter-studyvariation
– Accountsforinter-studyvariation• Metaregression– Accountforstudy-levelregressors– Fixedorrandomeffects
![Page 6: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/6.jpg)
NeuroimagingMeta-AnalysisApproaches(1)
• RegionofInterest– TraditionalMeta-Analysis,onmean%BOLD&stderr– Almostimpossibletodo• ROI-basedresultsrare(exception:PET)• DifferentROIsusedbydifferentauthors• Peak%BOLDuseless,duetovoodoobias
– Peakisoverly-optimisticestimateof%BOLDinROI
MNIx-axis
True%BOLD
Estimated%BOLD
![Page 7: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/7.jpg)
NeuroimagingMeta-AnalysisApproaches(2)
• Intensity-BasedMeta-Analysis(IBMA)–WithP/T/ZImagesonly• OnlyallowsFishers/Stouffers
–WithCOPE’sonly• Onlyallowsrandom-effectsmodelwithoutweights
– Can’tweightbysamplesize!
–WithCOPE’s&VARCOPES• FSL’sFEAT/FLAME is therandomeffectmetamodel!
– 2nd-levelFLAME:Combiningsubjects– 3rd-levelFLAME:Combiningstudies
• Allowsmeta-regression– Butimagedatararelyshared
BestpracticeJ
NotbestpracticeL
NotbestpracticeL
BadpracticeL
![Page 8: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/8.jpg)
NeuroimagingMeta-AnalysisApproaches(3)
• Coordinate-BasedMeta-Analysis(CBMA)– x,y,z locationsonly• ActivationLikelihoodEstimation(ALE)
• MultilevelKernelDensityAnalysis(MKDA)
– x,y,z andZ-value• SignedDifferenceMapping(SDM)
Turkeltaub etal.(2002).Meta-analysisofthefunctionalneuroanatomyofsingle-wordreading:methodandvalidation.NeuroImage,16(3),765–780.Eickhoff etal.(2009). Coordinate-basedactivationlikelihoodestimationmeta-analysisofneuroimagingdata:arandom-effectsapproachbasedonempiricalestimatesofspatialuncertainty.HumanBrainMapping,30(9),2907-26.Eickhoff etal.(2012). Activationlikelihoodestimationmeta-analysisrevisited.NeuroImage,59(3),2349–61
Wageretal.(2004). Neuroimagingstudiesofshiftingattention:ameta-analysis.NeuroImage 22(4),1679–1693.Kober etal.(2008). Functionalgroupingandcortical-subcorticalinteractionsinemotion:ameta-analysisofneuroimagingstudies.NeuroImage,42(2),998–1031.
Radua &Mataix-Cols(2009). Voxel-wisemeta-analysisofgreymatterchangesinobsessive-compulsivedisorder.BritishJournalofPsychiatry,195:391-400.Costafreda etal.(2009). Aparametricapproachtovoxel- basedmeta-analysis.NeuroImage,46(1):115-122.
![Page 9: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/9.jpg)
CMBAKernelMethods• Createstudymaps– Eachfocusisreplacedwithkernel
• Importantdetailsonkerneloverlap
• Createmetamaps– Studymapscombined
• Inference– Traditionalvoxel-wiseorcluster-wise
• Voxel-wise– FDRorFWE• Cluster-wise– FWE
– MonteCarlotest• H0:noconsistencyoverstudies• Randomlyplaceeachstudy’sfoci,recreatemetamaps• Notactuallyapermutationtest(seeBesag&Diggle (1977))
Besag &Diggle (1977).SimpleMonteCarlotestsforspatialpattern.JRSSC(AppliedStatistics),26(3),327–333.
Wager etal.(2007).SCAN,2(2),150–8.
![Page 10: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/10.jpg)
Study 1Study 1Study 2Study 3
KernelMethodsHistory– m/ALE
Study 2Study 3
ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)
ALEper-studymap
ALEmapkernelFHWMf
ALEinterpretationforsinglefocus()Probabilityofobservingafocusatthatlocation()
ALEcombiningProbabilityofunionofevents…ALE(p1,p2)=p1+p2−p1×p2ALE(p1,p2,p3)=p1+p2+p3−p1×p2−p1×p3−p2×p3+ p1×p2×p3
ALEinterpretation:Probabilityofobservingoneormorefociatagivenlocationbased onamodelofGaussianspreadwithFWHMf
![Page 11: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/11.jpg)
Study 1Study 1Study 2Study 3
KernelMethodsHistory– m/ALE
Study 2Study 3
ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)
ALEper-studymap
ALEmapkernelFHWMf
ProblemwithfirstALESinglestudycoulddominate,iflotsonehaslotsofpoints
ModifiedALE(Eickhoff etal.,2009;Eickhoff etal.,2012)RevisedMonteCarlotestaccountsforstudies
Fixfoci,randomlysampleeachmapAdaptkernelsizef tostudysamplesizeVoxel-wisetest– noMonteCarlo!Cluster-wisetest– stillrequiresMonteCarlo
![Page 12: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/12.jpg)
Study 1Study 2Study 3
Study 1
MKDAmap – weightedaverageofstudymaps
Study 1Study 2Study 3
KernelMethodsHistory– M/KDA
SameproblemwithindividualprofligatestudiesMKDA(Kober etal.,2008)
TruncatedkernelMonteCarlotest
Movesclusters,notindividualfoci
MKDA(unweighted) interpretation:Proportionofstudieshavingoneormorefociwithindistancer
Study 2Study 3
KDA– KernelDensityAnalysis(Wageretal.,2004)
KDAper-studymap
KDAmap– averageofstudymaps
MKDA
MKDA– MultilevelKernelDensityAnalysisper-studymap
kernelradiusr
![Page 13: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/13.jpg)
CBMALimitations• Effectsize– Non-imagingMAisallabouteffectsize,CI’s–Whatistheeffectsize?• MKDA– Proportionofstudyresultinneighborhood• ALE– Probabilityatindividualvoxeloneorfoci
– Standarderrors?CI’s?– Power/sensitivity• 5/10studies– Great!• 5/100studies– Notgreat?Orsubtleevidence?
• Fixedvs.RandomEffects?
![Page 14: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/14.jpg)
• Aneffectthatgeneralizestothepopulationstudied
• Significancerelativetobetween-studyvariation
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
0
Distribution of each study’s estimated effect
Distribution of population effect
s2FFX
s2RFX
IBMARandomEffects?
%BOLD
![Page 15: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/15.jpg)
MNIx-axis
• CBMA– Aneffectthatgeneralizestothepopulationstudied?• 5/10signif.:OK?• 5/100signif.:OK!?
– Significancerelativetobetween-studyvariation?• Significancebasedonnullofrandomdistribution
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Location of each study’s foci
Intensity Functione.g. ALE
WhatisaRandomEffect?
… under Ho
![Page 16: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/16.jpg)
MNIx-axis
• BayesianHierarchicalMarkedSpatialindependentClusterProcess– Explicitlyparameterizesintra- andinter-studyvariation
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Intensity Function
s2Study
s2Population
WhatisaRandomEffect?
Location of each study’s foci
Kang,Johnson,Nichols,&Wage(2011).MetaAnalysisofFunctionalNeuroimagingDataviaBayesianSpatialPointProcesses.JournaloftheAmericanStatisticalAssociation,106(493),124–134.
![Page 17: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/17.jpg)
CBMASensitivityanalyses
Wageretal.(2009).Evaluatingtheconsistencyandspecificityofneuroimagingdatausingmeta-analysis.NeuroImage,45(1S1),210–221.
• Z-scoresshouldfalltozerowithsamplesize
• MetaDiagnostics– Variousplotsassesswhetherexpectedbehavioroccurs
![Page 18: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/18.jpg)
CBMAFileDrawerBias?• Whatabout“P<0.001uncorrected”bias?
• Forrestplot–MKDAvaluesforrightamygdala
– Canexploredifferentexplanationsfortheeffect
0 20 40 60 80Percent of studies reporting a foci
within 10mm of right amygdala
Chance: whole−brain FWE threshold
Chance: small−volume FWE threshold
Chance: half of all studiesusing P<0.001 uncorrected
Chance: all studiesusing P<0.001 uncorr.
Emotion Meta Analysis from 154 studiesRight Amygdala activation
Anger (26 studies)
Disgust (28 studies)
Fear (43 studies)
Happy (24 studies)
Sad (33 studies)
All (154 studies)
T.Nichols
![Page 19: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/19.jpg)
Foci per contrast
Den
sity
0 5 10 15 20 25
0.00
0.05
0.10
0.15
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
● ● ● ● ● ●
EstimatingSizeoftheFileDrawer• Estimationof“FileDrawer”prevalence
• Usefocicountstoinfernumberofmissing(0count)studies
• About1studymissingper10published– 9.02per10095%CI(7.32,10.72)
– Variesbysubarea
CountsPerContrastEmpirical&FittedDistribution
2,562StudiesfromBrainMapOnecontrastperstudyrandomlyselected
Pantelis Samartsidis,etal.OHBM2015Poster4038-W“Estimatingtheprevalenceof‘filedrawer’studies”
![Page 20: Overview of Meta-Analysis Approaches](https://reader036.fdocuments.in/reader036/viewer/2022081503/62a1f970094aab6d6f4aea0d/html5/thumbnails/20.jpg)
Conclusions
• IBMA–Wouldbegreat,richtoolsavailable
• CBMA– 2+toolsavailable– Stilllotsofworktodeliverbest(statistical)practicetoinferences