Statistical Inference, Multiple ComparisonsStatistical Inference, Multiple Comparisonsand Random Field Theoryand Random Field Theory

Statistical Inference, Multiple ComparisonsStatistical Inference, Multiple Comparisonsand Random Field Theoryand Random Field Theory

Andrew HolmesAndrew HolmesSPM short course, May 2002SPM short course, May 2002

Andrew HolmesAndrew HolmesSPM short course, May 2002SPM short course, May 2002

……a voxel by voxel hypothesis testing approacha voxel by voxel hypothesis testing approach reliably identify regions showing a reliably identify regions showing a

significant experimental effect of interestsignificant experimental effect of interest

• Assessment of statistic imagesAssessment of statistic images• multiple comparisonsmultiple comparisons• random field theoryrandom field theory• smoothnesssmoothness• spatial levels of inference & powerspatial levels of inference & power• false discovery ratefalse discovery rate later... later...

• Generalisability, random effects Generalisability, random effects & population inference& population inference

• inferring to the populationinferring to the population• group comparisonsgroup comparisons

• Non-parametric inference Non-parametric inference later... later...

Overview…Overview…Overview…Overview…

realignment &motion

correction

smoothing

normalisation

General Linear Modelmodel fittingstatistic image

corrected p-values

image data parameterestimatesdesign

matrix

anatomicalreference

kernel

StatisticalParametric Map

random field theory

Statistical Parametric Mapping…Statistical Parametric Mapping…Statistical Parametric Mapping…Statistical Parametric Mapping…

gCBF

rCB

F

x

o

o

o

o

o

o

x

x

x

x

x

g..

k1

k2

k

condition 1 condition 2

voxel by voxelmodelling

–

parameter estimate variance estimate

=

statistic imageor

SPM

Classical hypothesis testing…Classical hypothesis testing…Classical hypothesis testing…Classical hypothesis testing…

• Null hypothesis Null hypothesis HH– test statistictest statistic– null distributionsnull distributions

• Hypothesis testHypothesis test– control Type I errorcontrol Type I error

• incorrectly reject incorrectly reject HH

– test test levellevel • Pr(“reject” Pr(“reject” HH | | HH) )

– test test sizesize• Pr(“reject Pr(“reject HH | | HH))

• p p –value–value– min min at which at which HH rejectedrejected– Pr(Pr(T T   tt |  | HH))– characterising characterising “surprise”“surprise”

• Null hypothesis Null hypothesis HH– test statistictest statistic– null distributionsnull distributions

• Hypothesis testHypothesis test– control Type I errorcontrol Type I error

• incorrectly reject incorrectly reject HH

– test test levellevel • Pr(“reject” Pr(“reject” HH | | HH) )

– test test sizesize• Pr(“reject Pr(“reject HH | | HH))

• p p –value–value– min min at which at which HH rejectedrejected– Pr(Pr(T T   tt |  | HH))– characterising characterising “surprise”“surprise”

t –distribution, 32 df.

F –distribution, 10,32 df.

Multiple comparisons…Multiple comparisons…Multiple comparisons…Multiple comparisons…

t59

Gaussian10mm FWHM(2mm pixels)

p = 0.05

• Threshold at Threshold at p p ??– expect (100 expect (100 pp)% by chance)% by chance

• Surprise Surprise ??– extreme voxel valuesextreme voxel values

voxel level inferencevoxel level inference

– big suprathreshold clustersbig suprathreshold clusters cluster level inferencecluster level inference

– many suprathreshold clustersmany suprathreshold clusters set level inferenceset level inference

• Power & localisationPower & localisation sensitivitysensitivity spatial specificityspatial specificity

• Threshold at Threshold at p p ??– expect (100 expect (100 pp)% by chance)% by chance

• Surprise Surprise ??– extreme voxel valuesextreme voxel values

voxel level inferencevoxel level inference

– big suprathreshold clustersbig suprathreshold clusters cluster level inferencecluster level inference

– many suprathreshold clustersmany suprathreshold clusters set level inferenceset level inference

• Power & localisationPower & localisation sensitivitysensitivity spatial specificityspatial specificity

• FamilyFamily of hypotheses of hypotheses– HHk k k k = {1,…, = {1,…,KK}}

– HH = = HHkk

• FamilywiseFamilywise Type I error Type I error– weakweak control – control – omnibus testomnibus test

• Pr(“reject” Pr(“reject” HH HH) ) • ““anything, anywhere”anything, anywhere” ??

– strongstrong control – control – localising testlocalising test

• Pr(“reject” HPr(“reject” HWW H HWW) ) W: W W: W & H & HWW

• ““anything, & where”anything, & where” ??

• Adjusted Adjusted pp–values–values– test level at which reject test level at which reject HHkk

• FamilyFamily of hypotheses of hypotheses– HHk k k k = {1,…, = {1,…,KK}}

– HH = = HHkk

• FamilywiseFamilywise Type I error Type I error– weakweak control – control – omnibus testomnibus test

• Pr(“reject” Pr(“reject” HH HH) ) • ““anything, anywhere”anything, anywhere” ??

– strongstrong control – control – localising testlocalising test

• Pr(“reject” HPr(“reject” HWW H HWW) ) W: W W: W & H & HWW

• ““anything, & where”anything, & where” ??

• Adjusted Adjusted pp–values–values– test level at which reject test level at which reject HHkk

Multiple comparisons terminology…Multiple comparisons terminology…Multiple comparisons terminology…Multiple comparisons terminology…

p = 0.05

p = 0.0000001

p = 0.0001

Simple threshold tests…Simple threshold tests…Simple threshold tests…Simple threshold tests…

• Threshold Threshold u u 

– ttkk > > uu reject reject HHkk

– reject any reject any HHkk reject reject HH

reject reject HH if if ttmaxmax > > uu

• Valid testValid test– weakweak control control

Pr(Pr(TTmaxmax > > uu HH

  ) )

– strongstrong control controlsince W since W Pr(Pr(TTWW

maxmax > > uu HHWW  ) )

• Adjusted Adjusted pp –values –values– Pr(Pr(TT

maxmax > > ttkk HH))

• Threshold Threshold u u 

– ttkk > > uu reject reject HHkk

– reject any reject any HHkk reject reject HH

reject reject HH if if ttmaxmax > > uu

• Valid testValid test– weakweak control control

Pr(Pr(TTmaxmax > > uu HH

  ) )

– strongstrong control controlsince W since W Pr(Pr(TTWW

maxmax > > uu HHWW  ) )

• Adjusted Adjusted pp –values –values– Pr(Pr(TT

maxmax > > ttkk HH)) uuuu

'Pr1 KAk

kk AA PrPr1

The “Bonferroni” correction…The “Bonferroni” correction…The “Bonferroni” correction…The “Bonferroni” correction…

• ““The” Bonferroni inequalityThe” Bonferroni inequalityCarlo Emilio Bonferroni (1936)Carlo Emilio Bonferroni (1936)

– For any set of events AFor any set of events Akk : :

• Bonferroni correctionBonferroni correction– AAkk : correctly “accept” H : correctly “accept” Hkk

TTkk < < uu & H & Hkk

– Assess HAssess Hkk at level at level ''

correction correction ' = ' = / / KK

• Adjusted Adjusted pp –values –values– min(1,min(1,KK ppk k ))

• ““The” Bonferroni inequalityThe” Bonferroni inequalityCarlo Emilio Bonferroni (1936)Carlo Emilio Bonferroni (1936)

– For any set of events AFor any set of events Akk : :

• Bonferroni correctionBonferroni correction– AAkk : correctly “accept” H : correctly “accept” Hkk

TTkk < < uu & H & Hkk

– Assess HAssess Hkk at level at level ''

correction correction ' = ' = / / KK

• Adjusted Adjusted pp –values –values– min(1,min(1,KK ppk k ))

Conservative for correlated testsConservative for correlated tests

independent:independent: KK tests tests

some dependence :some dependence : ?? tests tests

totally dependent:totally dependent: 1 test1 test

Conservative for correlated testsConservative for correlated tests

independent:independent: KK tests tests

some dependence :some dependence : ?? tests tests

totally dependent:totally dependent: 1 test1 test

u = -1(1-/K)

5mm

10mm

15mm

SPM approach: Random fields…SPM approach: Random fields…SPM approach: Random fields…SPM approach: Random fields…

• Consider statistic image as lattice representation of a Consider statistic image as lattice representation of a continuous random fieldcontinuous random field

• Use results from continuous random field theoryUse results from continuous random field theory

• Consider statistic image as lattice representation of a Consider statistic image as lattice representation of a continuous random fieldcontinuous random field

• Use results from continuous random field theoryUse results from continuous random field theory

lattice represtntation

Euler characteristic…Euler characteristic…Euler characteristic…Euler characteristic…

• Topological measureTopological measure– of excursion set of excursion set uu

AAuu = {x RR33 : Z(x) > u}

– # components - # “holes”# components - # “holes”

• Single threshold testSingle threshold test– large large uu, near , near TTmaxmax

– Euler char. Euler char. #local max #local max– Expected Euler char Expected Euler char pp–value–value

Pr(Pr(ZZmaxmax > > u u ) )

Pr( Pr(uu) > 0 ) > 0 )

EE[[uu)]]

– single threshold testsingle threshold test

– uu s.t. s.t. EE[[uu) ] = ] =

• Topological measureTopological measure– of excursion set of excursion set uu

AAuu = {x RR33 : Z(x) > u}

– # components - # “holes”# components - # “holes”

• Single threshold testSingle threshold test– large large uu, near , near TTmaxmax

– Euler char. Euler char. #local max #local max– Expected Euler char Expected Euler char pp–value–value

Pr(Pr(ZZmaxmax > > u u ) )

Pr( Pr(uu) > 0 ) > 0 )

EE[[uu)]]

– single threshold testsingle threshold test

– uu s.t. s.t. EE[[uu) ] = ] =

This EPS image does not contain a screen preview.It will print correctly to a PostScript printer.File Name : Euler_Char.epsTitle : Euler_Char.epsCreator : MATLAB, The Mathworks, Inc.CreationDate : 03/04/96 16:54:29Pages : 1This EPS image does not contain a screen preview.It will print correctly to a PostScript printer.File Name : Euler_Char.epsTitle : Euler_Char.epsCreator : MATLAB, The Mathworks, Inc.CreationDate : 03/04/96 16:54:29Pages : 1

EE[[uu)] ] (()) | ||| (u u 2 2 -1-1) exp exp(--u u 22/2/2) / (22)22

– largelarge search regionsearch region RR3 3

– (( volumevolume |||| smoothnesssmoothness

– AAuu excursion setexcursion set AAuu = {x RR33 : Z(x) > u}

– Z(x) Gaussian random fieldGaussian random field x RR33

+ Multivariate Normal Finite Dimensional + Multivariate Normal Finite Dimensional distributionsdistributions

+ continuous+ continuous+ strictly stationary+ strictly stationary+ marginal N+ marginal N((0,10,1))+ continuously differentiable+ continuously differentiable+ twice differentiable at 0+ twice differentiable at 0+ Gaussian + Gaussian ACFACF (at least near local maxima)(at least near local maxima)

EE[[uu)] ] (()) | ||| (u u 2 2 -1-1) exp exp(--u u 22/2/2) / (22)22

– largelarge search regionsearch region RR3 3

– (( volumevolume |||| smoothnesssmoothness

– AAuu excursion setexcursion set AAuu = {x RR33 : Z(x) > u}

– Z(x) Gaussian random fieldGaussian random field x RR33

+ Multivariate Normal Finite Dimensional + Multivariate Normal Finite Dimensional distributionsdistributions

+ continuous+ continuous+ strictly stationary+ strictly stationary+ marginal N+ marginal N((0,10,1))+ continuously differentiable+ continuously differentiable+ twice differentiable at 0+ twice differentiable at 0+ Gaussian + Gaussian ACFACF (at least near local maxima)(at least near local maxima)

Expected Euler characteristic…Expected Euler characteristic…Expected Euler characteristic…Expected Euler characteristic…

Au

ze

ze

ye

ze

xe

ze

ye

ye

ye

xe

ze

xe

ye

xe

xe

var,cov,cov

,covvar,cov

,cov,covvar

Smoothness, PRF, resels...Smoothness, PRF, resels...Smoothness, PRF, resels...Smoothness, PRF, resels...

• Smoothness Smoothness |||| – variance-covariance matrix of partial variance-covariance matrix of partial

derivatives derivatives (possibly location dependent)(possibly location dependent)

• Point Response Function Point Response Function PRFPRF

• Full Width at Half Maximum Full Width at Half Maximum FWHMFWHM

• Smoothness Smoothness |||| – variance-covariance matrix of partial variance-covariance matrix of partial

derivatives derivatives (possibly location dependent)(possibly location dependent)

• Point Response Function Point Response Function PRFPRF

• Full Width at Half Maximum Full Width at Half Maximum FWHMFWHM

• Gaussian Gaussian PRFPRF– – – kernel var/cov matrixkernel var/cov matrix– ACFACF 2 2 – = (2= (2))-1-1

FWHMFWHM f = f = (8ln(2))(8ln(2))

ffxx 0 0– ffyy0 1

0 0 ffzz 8ln(2)ignoring covariancesignoring covariances

|||| = (4ln(2)) = (4ln(2))3/23/2 / (f / (fxx f fyy f fzz))

• ResResolution olution ElElement (ement (RESELRESEL))– Resel dimensions (fResel dimensions (fxx f fyy f fzz))– RR33(() = ) = (()) / (f / (fxx f fyy f fzz))

if strictly stationaryif strictly stationary

EE[[uu)]] = R = R33(() (4ln(2))) (4ln(2))3/23/2 ( (u u 2 2 -1) exp(--1) exp(-u u 22/2) / (2/2) / (2))22

RR33(() ) ((1 – 1 – (u)(u)))for high thresholds ufor high thresholds u

• Gaussian Gaussian PRFPRF– – – kernel var/cov matrixkernel var/cov matrix– ACFACF 2 2 – = (2= (2))-1-1

FWHMFWHM f = f = (8ln(2))(8ln(2))

ffxx 0 0– ffyy0 1

0 0 ffzz 8ln(2)ignoring covariancesignoring covariances

|||| = (4ln(2)) = (4ln(2))3/23/2 / (f / (fxx f fyy f fzz))

• ResResolution olution ElElement (ement (RESELRESEL))– Resel dimensions (fResel dimensions (fxx f fyy f fzz))– RR33(() = ) = (()) / (f / (fxx f fyy f fzz))

if strictly stationaryif strictly stationary

EE[[uu)]] = R = R33(() (4ln(2))) (4ln(2))3/23/2 ( (u u 2 2 -1) exp(--1) exp(-u u 22/2) / (2/2) / (2))22

RR33(() ) ((1 – 1 – (u)(u)))for high thresholds ufor high thresholds u

Component fields…Component fields…Component fields…Component fields…

= +Y X

data matrix

des

ign

mat

rix

parameters errors+ ?= ?voxelsvoxels

scansscans

estimate

^

residuals

estimatedcomponent

fields

parameterestimates

“Image regression”

variance

estimated variance

=

Smoothness estimation…Smoothness estimation…Smoothness estimation…Smoothness estimation…

• SmoothnessSmoothness– from standardised residualsfrom standardised residuals

– empirical derivatives at each voxel empirical derivatives at each voxel • Resels per voxel (RPV) – an “image” of smoothnessResels per voxel (RPV) – an “image” of smoothness

– correction for estimation of variance field correction for estimation of variance field 22

• function of degrees of freedomfunction of degrees of freedom

– covariances often ignoredcovariances often ignored

• Euler CharacteristicsEuler Characteristics– using discrete methodsusing discrete methods

• SmoothnessSmoothness– from standardised residualsfrom standardised residuals

– empirical derivatives at each voxel empirical derivatives at each voxel • Resels per voxel (RPV) – an “image” of smoothnessResels per voxel (RPV) – an “image” of smoothness

– correction for estimation of variance field correction for estimation of variance field 22

• function of degrees of freedomfunction of degrees of freedom

– covariances often ignoredcovariances often ignored

• Euler CharacteristicsEuler Characteristics– using discrete methodsusing discrete methods

• General form for expected Euler characteristicGeneral form for expected Euler characteristic• 22, , FF, & , & tt fields fields •• restricted search regions restricted search regions •• DD dimensions dimensions ••

EE[[(AAuu)] = ] = R Rd d (()) d d ((uu))

• General form for expected Euler characteristicGeneral form for expected Euler characteristic• 22, , FF, & , & tt fields fields •• restricted search regions restricted search regions •• DD dimensions dimensions ••

EE[[(AAuu)] = ] = R Rd d (()) d d ((uu))

Unified Unified pp-values…-values…Unified Unified pp-values…-values…

Rd (): d-dimensional Minkowskifunctional of

– function of dimension,space and smoothness:

R0() = () Euler characteristic of

R1() = resel diameter

R2() = resel surface area

R3() = resel volume

d (): d-dimensional EC density of Z(x)– function of dimension and threshold,

specific for RF type:

E.g. Gaussian RF: (strictly stationary &c…)

0(u) = 1- (u)

1(u) = (4 ln2)1/2 exp(-u2/2) / (2)

2(u) = (4 ln2) exp(-u2/2) / (2)3/2

3(u) = (4 ln2)3/2 (u2 -1) exp(-u2/2) / (2)2

4(u) = (4 ln2)2 (u3 -3u) exp(-u2/2) / (2)5/2

Au

Suprathreshold cluster tests…Suprathreshold cluster tests…Suprathreshold cluster tests…Suprathreshold cluster tests…

• Primary threshold Primary threshold uu– examine connected components examine connected components

of excursion setof excursion set

– Suprathreshold clustersSuprathreshold clusters

– Reject Reject HHWW for clusters of voxels for clusters of voxels WW of size of size SS > > ss

• Localisation Localisation (Strong control)(Strong control)

– at cluster levelat cluster level

– increased powerincreased power– esp. high resolutions esp. high resolutions ((ff MRI MRI))

• Thresholds, Thresholds, pp –values –values– Pr(Pr(SS

maxmax > > ss H H ) )

Nosko, Friston, (Worsley)Nosko, Friston, (Worsley)

– Poisson occurrence Poisson occurrence (Adler)(Adler)

– Assumme form for Pr(Assumme form for Pr(SS==ss||SS>0)>0)

• Primary threshold Primary threshold uu– examine connected components examine connected components

of excursion setof excursion set

– Suprathreshold clustersSuprathreshold clusters

– Reject Reject HHWW for clusters of voxels for clusters of voxels WW of size of size SS > > ss

• Localisation Localisation (Strong control)(Strong control)

– at cluster levelat cluster level

– increased powerincreased power– esp. high resolutions esp. high resolutions ((ff MRI MRI))

• Thresholds, Thresholds, pp –values –values– Pr(Pr(SS

maxmax > > ss H H ) )

Nosko, Friston, (Worsley)Nosko, Friston, (Worsley)

– Poisson occurrence Poisson occurrence (Adler)(Adler)

– Assumme form for Pr(Assumme form for Pr(SS==ss||SS>0)>0)

5mm FWHM

10mm FWHM

15mm FWHM

(2mm2 pixels)

Levels of inference…Levels of inference…Levels of inference…Levels of inference…

ParametersParametersu u - - 3.093.09k k - - 12 voxels12 voxelsS S - - 32323 3 voxelsvoxelsFWHMFWHM - - 4.7 voxels4.7 voxelsD D - - 33

set-levelset-levelP(c P(c 3 | n 3 | n 12, u 12, u 3.09) = 3.09) =

0.0190.019

cluster-levelcluster-levelP(c P(c 1 | n 1 | n 82, t 82, t 3.09) = 0.029 (corrected) 3.09) = 0.029 (corrected)P(n P(n 82 | t 82 | t 3.09) = 0.019 (uncorrected) 3.09) = 0.019 (uncorrected)

n=82n=82

n=32n=32

n=1n=122

voxel-levelvoxel-levelP(c P(c 1 | n 1 | n 0, t 0, t 4.37) = 0.048 (corrected) 4.37) = 0.048 (corrected)P(P(tt 4.37) = 1 - 4.37) = 1 - {{4.374.37} } < 0.001 (uncorrected)< 0.001 (uncorrected) omnibusomnibus

P(cP(c7 | n 7 | n 0, u 0, u 3.09) = 0.031 3.09) = 0.031

This EPS image does not contain a screen preview.It will print correctly to a PostScript printer.File Name : level_inf.epsTitle : level_inf.epsCreator : CLARIS EPSF Export Filter V1.0CreationDate : 5/11/96 5:44:15 p.m.This EPS image does not contain a screen preview.It will print correctly to a PostScript printer.File Name : recap_tests.epsTitle : recap_tests.epsCreator : CLARIS EPSF Export Filter V1.0CreationDate : 5/12/96 2:13:30 p.m.Summary: Levels of inference & powerSummary: Levels of inference & powerSummary: Levels of inference & powerSummary: Levels of inference & power

SPM results...SPM results...SPM results...SPM results...

SPM results...SPM results...SPM results...SPM results...

SPM results...SPM results...SPM results...SPM results...

SPM results...SPM results...SPM results...SPM results...

Assumptions…Assumptions…Assumptions…Assumptions…

• Model fit & assumptionsModel fit & assumptions– valid distributional resultsvalid distributional results

• Multivariate normalityMultivariate normality– of of componentcomponent images images

• Strict stationarity Strict stationarity (pre SPM99)(pre SPM99)

– of of componentcomponent images images

– homogeneous spatial structurehomogeneous spatial structure

• Model fit & assumptionsModel fit & assumptions– valid distributional resultsvalid distributional results

• Multivariate normalityMultivariate normality– of of componentcomponent images images

• Strict stationarity Strict stationarity (pre SPM99)(pre SPM99)

– of of componentcomponent images images

– homogeneous spatial structurehomogeneous spatial structure

• SmoothnessSmoothness– smoothness » voxel sizesmoothness » voxel size

• lattice approximationlattice approximation• smoothness estimationsmoothness estimation

– practicallypractically• FWHMFWHM 3 3 VoxDimVoxDim

– otherwiseotherwise• conservativeconservative

(voxel level)(voxel level)

• laxlax(spatial extent)(spatial extent)

spatial smoothingspatial smoothing??temporal smoothingtemporal smoothing??

• SmoothnessSmoothness– smoothness » voxel sizesmoothness » voxel size

• lattice approximationlattice approximation• smoothness estimationsmoothness estimation

– practicallypractically• FWHMFWHM 3 3 VoxDimVoxDim

– otherwiseotherwise• conservativeconservative

(voxel level)(voxel level)

• laxlax(spatial extent)(spatial extent)

spatial smoothingspatial smoothing??temporal smoothingtemporal smoothing??

Subject1 Subject2 Subject3

• Fixed effectsFixed effects– Are you confident that a new Are you confident that a new

observation from any of subjects observation from any of subjects 1-3 will be greater than zero1-3 will be greater than zero??

• Yes!Yes!using within-subjects varianceusing within-subjects variance

– infer for these subjects – infer for these subjects – case studycase study

• Random effectsRandom effects– Are you confident that a new Are you confident that a new

observation from a new subject will observation from a new subject will be greater than zerobe greater than zero??

• No!No!using between-subjects varianceusing between-subjects variance

– infer for any subject infer for any subject – – populationpopulation

• Fixed effectsFixed effects– Are you confident that a new Are you confident that a new

observation from any of subjects observation from any of subjects 1-3 will be greater than zero1-3 will be greater than zero??

• Yes!Yes!using within-subjects varianceusing within-subjects variance

– infer for these subjects – infer for these subjects – case studycase study

• Random effectsRandom effects– Are you confident that a new Are you confident that a new

observation from a new subject will observation from a new subject will be greater than zerobe greater than zero??

• No!No!using between-subjects varianceusing between-subjects variance

– infer for any subject infer for any subject – – populationpopulation

Random effects & variance componentsRandom effects & variance componentsRandom effects & variance componentsRandom effects & variance components

Multi-subject analysis…?Multi-subject analysis…?Multi-subject analysis…?Multi-subject analysis…?

p < 0.001 (uncorrected)

p < 0.05 (corrected)

SPM{t}

SPM{t}

1^

2^

3^

4^

5^

6^

^

• – c.f. 2 / nw

—̂

^

^

^

^

^

– c.f.

estimated mean activation image

^

Two-stage analysis of random effect…Two-stage analysis of random effect…Two-stage analysis of random effect…Two-stage analysis of random effect…

1^

2^

3^

4^

5^

6^

^

^

^

^

^

• – c.f. 2/n = 2 /n + 2

/ nw^

– c.f.

level-one(within-subject)

variance 2^

an estimate of the mixed-effects model

variance 2

+ 2 / w

—

level-two(between-subject)

timecourses at [ 03, -78, 00 ] contrast images

p < 0.001 (uncorrected)

SPM{t}

(no voxels significant at p < 0.05 (corrected))

Two stage random effects group comparisonTwo stage random effects group comparisonTwo stage random effects group comparisonTwo stage random effects group comparison

12 subjects

contrast imageslevel-one(within-subject)

level-two(between-subject)

two-

sam

ple

t-te

st

vs.

Multi-stage multi-level modelling…Multi-stage multi-level modelling…Multi-stage multi-level modelling…Multi-stage multi-level modelling…

parameter estimation inference

level-1 data, model & contrast(s)

estimated contrasts from level-1 fits,level-2 model & level-2 contrasts

level 2 estimated contrasts and

residual variance

level 2(population)

inference

Hypothesis testing Hypothesis testing !!??Hypothesis testing Hypothesis testing !!??

• Why testWhy test??• reliability reliability genuine effects genuine effects integrity of research integrity of research (hopefully) (hopefully)

• The fallacy…The fallacy…• point null hypothesispoint null hypothesis

(no change)(no change)• things are never the samethings are never the same! !

(always some small chance change)(always some small chance change)• given enough observations given enough observations

can always reject null hypothesis can always reject null hypothesis !!• fMRI fMRI !!?? (lots of observations)(lots of observations)

……testing, rather than estimatingtesting, rather than estimating• significant significant important important !!??

……and:and:““absence of evidence absence of evidence

isis notnot evidence of absenceevidence of absence””

!?

Worsley KJ, Marrett S, Neelin P, Evans AC (1992)

“A three-dimensional statistical analysis for CBF activation studies in human brain”Journal of Cerebral Blood Flow and Metabolism 12:900-918

Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC (1995)

“A unified statistical approach for determining significant signals in images of cerebral activation”Human Brain Mapping 4:58-73

Friston KJ, Worsley KJ, Frackowiak RSJ, Mazziotta JC, Evans AC (1994)“Assessing the Significance of Focal Activations Using their Spatial Extent”Human Brain Mapping 1:214-220

Cao J (1999)“The size of the connected components of excursion sets of 2, t and F fields”Advances in Applied Probability (in press)

Worsley KJ, Marrett S, Neelin P, Evans AC (1995)“Searching scale space for activation in PET images”Human Brain Mapping 4:74-90

Worsley KJ, Poline J-B, Vandal AC, Friston KJ (1995)“Tests for distributed, non-focal brain activations”NeuroImage 2:183-194

Friston KJ, Holmes AP, Poline J-B, Price CJ, Frith CD (1996)“Detecting Activations in PET and fMRI: Levels of Inference and Power”Neuroimage 4:223-235

Worsley KJ, Marrett S, Neelin P, Evans AC (1992)

“A three-dimensional statistical analysis for CBF activation studies in human brain”Journal of Cerebral Blood Flow and Metabolism 12:900-918

Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC (1995)

“A unified statistical approach for determining significant signals in images of cerebral activation”Human Brain Mapping 4:58-73

Friston KJ, Worsley KJ, Frackowiak RSJ, Mazziotta JC, Evans AC (1994)“Assessing the Significance of Focal Activations Using their Spatial Extent”Human Brain Mapping 1:214-220

Cao J (1999)“The size of the connected components of excursion sets of 2, t and F fields”Advances in Applied Probability (in press)

Worsley KJ, Marrett S, Neelin P, Evans AC (1995)“Searching scale space for activation in PET images”Human Brain Mapping 4:74-90

Worsley KJ, Poline J-B, Vandal AC, Friston KJ (1995)“Tests for distributed, non-focal brain activations”NeuroImage 2:183-194

Friston KJ, Holmes AP, Poline J-B, Price CJ, Frith CD (1996)“Detecting Activations in PET and fMRI: Levels of Inference and Power”Neuroimage 4:223-235

Multiple Comparisons,Multiple Comparisons,& Random Field Theory& Random Field TheoryMultiple Comparisons,Multiple Comparisons,

& Random Field Theory& Random Field Theory

Ch5 Ch4

indexindexindexindex

• overviewoverview• multiple comparisonsmultiple comparisons• random field theoryrandom field theory• random effectsrandom effects• hypothesis testing fallacyhypothesis testing fallacy

• overviewoverview• multiple comparisonsmultiple comparisons• random field theoryrandom field theory• random effectsrandom effects• hypothesis testing fallacyhypothesis testing fallacy