Post on 09-Jun-2022
OverviewofMeta-AnalysisApproaches
ThomasE.NicholsUniversityofOxford
NeuroimagingMeta-AnalysisOHBMEducationalCourse
25June,2017slides&posters@http://warwick.ac.uk/tenichols/ohbm
Overview
• Non-imagingmeta-analysis• Menuofmeta-analysismethods– ROI’s,IBMA,CBMA
• CBMAdetails– Kernel-basedmethods– What’sincommon– m/ALE,M/KDA– What’sdifferent
• Limitations&Thoughts
Stagesof(non-imaging) Meta-Analysis
1. Definereview'sspecificobjectives.2. Specifyeligibilitycriteria.3. Identifyalleligiblestudies.4. Collectandvalidatedatarigorously.5. Displayeffectsforeachstudy,withmeasuresof
precision.6. Computeaverageeffect,randomeffectsstd err7. Checkforpublicationbias,conductsensitivity
analyses.
Jones,D.R.(1995).Meta-analysis:weighingtheevidence.StatisticsinMedicine,14(2),137–49.
Methodsfor(non-imaging) Meta-Analysis(1)• P-value(orZ-value)combining– Fishers(≈average–logP)– Stouffers(≈averageZ)– Usedonlyasmethodoflastresort
• Basedonsignificance,noteffectsinrealunits• Differingn willinduceheterogeneity (Cummings,2004)
• Fixedeffectsmodel– Requireseffectestimatesandstandarderrors
• E.g.Meansurvival(days),andstandarderrorofmean– Givesweightedaverageofeffects
• Weightsbasedonper-studystandarderrors– Neglectsinter-studyvariation
Cummings(2004).Meta-analysisbasedonstandardizedeffectsisunreliable.ArchivesofPediatrics&AdolescentMedicine,158(6),595–7.
Methodsfor(non-imaging) Meta-Analysis(2)• Randomeffectsmodel– Requireseffectestimatesandstandarderrors– Givesweightedaverageofeffect• Weightsbasedonper-studystandarderrorsandinter-studyvariation
– Accountsforinter-studyvariation• Metaregression– Accountforstudy-levelregressors– Fixedorrandomeffects
NeuroimagingMeta-AnalysisApproaches(1)
• RegionofInterest– TraditionalMeta-Analysis,onmean%BOLD&stderr– Almostimpossibletodo• ROI-basedresultsrare(exception:PET)• DifferentROIsusedbydifferentauthors• Peak%BOLDuseless,duetovoodoobias
– Peakisoverly-optimisticestimateof%BOLDinROI
MNIx-axis
True%BOLD
Estimated%BOLD
NeuroimagingMeta-AnalysisApproaches(2)
• Intensity-BasedMeta-Analysis(IBMA)–WithP/T/ZImagesonly• OnlyallowsFishers/Stouffers
–WithCOPE’sonly• Onlyallowsrandom-effectsmodelwithoutweights
– Can’tweightbysamplesize!
–WithCOPE’s&VARCOPES• FSL’sFEAT/FLAME is therandomeffectmetamodel!
– 2nd-levelFLAME:Combiningsubjects– 3rd-levelFLAME:Combiningstudies
• Allowsmeta-regression– Butimagedatararelyshared
BestpracticeJ
NotbestpracticeL
NotbestpracticeL
BadpracticeL
NeuroimagingMeta-AnalysisApproaches(3)
• Coordinate-BasedMeta-Analysis(CBMA)– x,y,z locationsonly• ActivationLikelihoodEstimation(ALE)
• MultilevelKernelDensityAnalysis(MKDA)
– x,y,z andZ-value• SignedDifferenceMapping(SDM)
Turkeltaub etal.(2002).Meta-analysisofthefunctionalneuroanatomyofsingle-wordreading:methodandvalidation.NeuroImage,16(3),765–780.Eickhoff etal.(2009). Coordinate-basedactivationlikelihoodestimationmeta-analysisofneuroimagingdata:arandom-effectsapproachbasedonempiricalestimatesofspatialuncertainty.HumanBrainMapping,30(9),2907-26.Eickhoff etal.(2012). Activationlikelihoodestimationmeta-analysisrevisited.NeuroImage,59(3),2349–61
Wageretal.(2004). Neuroimagingstudiesofshiftingattention:ameta-analysis.NeuroImage 22(4),1679–1693.Kober etal.(2008). Functionalgroupingandcortical-subcorticalinteractionsinemotion:ameta-analysisofneuroimagingstudies.NeuroImage,42(2),998–1031.
Radua &Mataix-Cols(2009). Voxel-wisemeta-analysisofgreymatterchangesinobsessive-compulsivedisorder.BritishJournalofPsychiatry,195:391-400.Costafreda etal.(2009). Aparametricapproachtovoxel- basedmeta-analysis.NeuroImage,46(1):115-122.
CMBAKernelMethods• Createstudymaps– Eachfocusisreplacedwithkernel
• Importantdetailsonkerneloverlap
• Createmetamaps– Studymapscombined
• Inference– Traditionalvoxel-wiseorcluster-wise
• Voxel-wise– FDRorFWE• Cluster-wise– FWE
– MonteCarlotest• H0:noconsistencyoverstudies• Randomlyplaceeachstudy’sfoci,recreatemetamaps• Notactuallyapermutationtest(seeBesag&Diggle (1977))
Besag &Diggle (1977).SimpleMonteCarlotestsforspatialpattern.JRSSC(AppliedStatistics),26(3),327–333.
Wager etal.(2007).SCAN,2(2),150–8.
Study 1Study 1Study 2Study 3
KernelMethodsHistory– m/ALE
Study 2Study 3
ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)
ALEper-studymap
ALEmapkernelFHWMf
ALEinterpretationforsinglefocus()Probabilityofobservingafocusatthatlocation()
ALEcombiningProbabilityofunionofevents…ALE(p1,p2)=p1+p2−p1×p2ALE(p1,p2,p3)=p1+p2+p3−p1×p2−p1×p3−p2×p3+ p1×p2×p3
ALEinterpretation:Probabilityofobservingoneormorefociatagivenlocationbased onamodelofGaussianspreadwithFWHMf
Study 1Study 1Study 2Study 3
KernelMethodsHistory– m/ALE
Study 2Study 3
ALE– ActivationLikelihoodEstimation(Turkeltaub etal.,2002)
ALEper-studymap
ALEmapkernelFHWMf
ProblemwithfirstALESinglestudycoulddominate,iflotsonehaslotsofpoints
ModifiedALE(Eickhoff etal.,2009;Eickhoff etal.,2012)RevisedMonteCarlotestaccountsforstudies
Fixfoci,randomlysampleeachmapAdaptkernelsizef tostudysamplesizeVoxel-wisetest– noMonteCarlo!Cluster-wisetest– stillrequiresMonteCarlo
Study 1Study 2Study 3
Study 1
MKDAmap – weightedaverageofstudymaps
Study 1Study 2Study 3
KernelMethodsHistory– M/KDA
SameproblemwithindividualprofligatestudiesMKDA(Kober etal.,2008)
TruncatedkernelMonteCarlotest
Movesclusters,notindividualfoci
MKDA(unweighted) interpretation:Proportionofstudieshavingoneormorefociwithindistancer
Study 2Study 3
KDA– KernelDensityAnalysis(Wageretal.,2004)
KDAper-studymap
KDAmap– averageofstudymaps
MKDA
MKDA– MultilevelKernelDensityAnalysisper-studymap
kernelradiusr
CBMALimitations• Effectsize– Non-imagingMAisallabouteffectsize,CI’s–Whatistheeffectsize?• MKDA– Proportionofstudyresultinneighborhood• ALE– Probabilityatindividualvoxeloneorfoci
– Standarderrors?CI’s?– Power/sensitivity• 5/10studies– Great!• 5/100studies– Notgreat?Orsubtleevidence?
• Fixedvs.RandomEffects?
• Aneffectthatgeneralizestothepopulationstudied
• Significancerelativetobetween-studyvariation
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
0
Distribution of each study’s estimated effect
Distribution of population effect
s2FFX
s2RFX
IBMARandomEffects?
%BOLD
MNIx-axis
• CBMA– Aneffectthatgeneralizestothepopulationstudied?• 5/10signif.:OK?• 5/100signif.:OK!?
– Significancerelativetobetween-studyvariation?• Significancebasedonnullofrandomdistribution
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Location of each study’s foci
Intensity Functione.g. ALE
WhatisaRandomEffect?
… under Ho
MNIx-axis
• BayesianHierarchicalMarkedSpatialindependentClusterProcess– Explicitlyparameterizesintra- andinter-studyvariation
Study 1
Study 2
Study 3
Study 4
Study 5
Study 6
Intensity Function
s2Study
s2Population
WhatisaRandomEffect?
Location of each study’s foci
Kang,Johnson,Nichols,&Wage(2011).MetaAnalysisofFunctionalNeuroimagingDataviaBayesianSpatialPointProcesses.JournaloftheAmericanStatisticalAssociation,106(493),124–134.
CBMASensitivityanalyses
Wageretal.(2009).Evaluatingtheconsistencyandspecificityofneuroimagingdatausingmeta-analysis.NeuroImage,45(1S1),210–221.
• Z-scoresshouldfalltozerowithsamplesize
• MetaDiagnostics– Variousplotsassesswhetherexpectedbehavioroccurs
CBMAFileDrawerBias?• Whatabout“P<0.001uncorrected”bias?
• Forrestplot–MKDAvaluesforrightamygdala
– Canexploredifferentexplanationsfortheeffect
0 20 40 60 80Percent of studies reporting a foci
within 10mm of right amygdala
Chance: whole−brain FWE threshold
Chance: small−volume FWE threshold
Chance: half of all studiesusing P<0.001 uncorrected
Chance: all studiesusing P<0.001 uncorr.
Emotion Meta Analysis from 154 studiesRight Amygdala activation
Anger (26 studies)
Disgust (28 studies)
Fear (43 studies)
Happy (24 studies)
Sad (33 studies)
All (154 studies)
T.Nichols
Foci per contrast
Den
sity
0 5 10 15 20 25
0.00
0.05
0.10
0.15
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
● ● ● ● ● ●
EstimatingSizeoftheFileDrawer• Estimationof“FileDrawer”prevalence
• Usefocicountstoinfernumberofmissing(0count)studies
• About1studymissingper10published– 9.02per10095%CI(7.32,10.72)
– Variesbysubarea
CountsPerContrastEmpirical&FittedDistribution
2,562StudiesfromBrainMapOnecontrastperstudyrandomlyselected
Pantelis Samartsidis,etal.OHBM2015Poster4038-W“Estimatingtheprevalenceof‘filedrawer’studies”
Conclusions
• IBMA–Wouldbegreat,richtoolsavailable
• CBMA– 2+toolsavailable– Stilllotsofworktodeliverbest(statistical)practicetoinferences