WM GM HC CA PU PA AM AC VE0

20

40

60

80

***

***

***

**

***

***

******

***

Foreground−Background

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

Atlas-­‐Based  Under-­‐Segmenta1on  Chris&an  Wachinger1,2,  Polina  Golland1  

1Computer  Science  and  Ar&ficial  Intelligence  Lab  (CSAIL),  MIT              2MassachuseDs  General  Hospital,  Harvard  Medical  School  

Acknowledgement:  We  thank  Jason  Chang  and  Greg  Sharp.  This  work  was  supported  in  part  by  the  Na&onal  Alliance  for  Medical  Image  Compu&ng  (U54-­‐EB005149)  and  the  NeuroImaging  Analysis  Center  (P41-­‐EB015902).  

Introduc1on  •  Atlas-­‐based  segmenta&on  is  commonly  used  in  medical  image  analysis  •  Tendency  of  atlas-­‐based  segmenta&on  to  under-­‐segment  structures  •  Dice  overlap  and  Hausdorff  distance  do  not  measure  under-­‐segmenta&on  •  Wang,  Yushkevich  (2012)  proposed  deconvolu&on  of  label  maps  •  Contribu&ons:  

•  Volume  overlap  measures  to  quan&fy  under-­‐segmenta&on  •  Hypothesis:  asymmetry  in  single  organ  segmenta1on  as  cause    •  Genera&ve  model  of  background  to  correct  under-­‐segmenta&on  

Conclusions  •  Significant  bias  in  atlas-­‐based  segmenta&on  to  under-­‐segmenta&on  •  Asymmetry  in  foreground-­‐background  segmenta&on  as  new  hypothesis  •  Genera&ve  model  to  par&&on  background  reduces  under-­‐segmenta&on  

Under-­‐Segmenta1on  Measures  to  quan&fy  over-­‐  and  under-­‐segmenta&on:  

Results  •  Datasets:  •  16  heart  MRA  images  (0.6x0.6x1.5mm)  •  18  head  and  neck  CT  scans  (0.9x0.9x2.5mm)  •  39  brain  MRI  with  1mm  isotropic  resolu&on  

•  Comparison  to  deconvolu&on  (Wang,  2012)  •  Details:    •  DP-­‐GMM,  GMM,  DP-­‐means,  k-­‐means  •  Global  and  local  approach  •  Patch  size:  3  x  3  x  3  

WM GM HC CA PU PA AM AC VE0

20

40

60

80

*****

Multi−Organ

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

O(S, S) =|S \ S||S|

U(S, S) =|S \ S||S|

↵ ⇡

�1

N

Pix

zix

µk,⌃k

WM GM HC CA PU PA AM AC VE0

20

40

60

80

***

***

**

***

***

Comparison of over− and under−segmentation, Latent Labels

Ove

r− a

nd

un

de

r−se

gm

en

tatio

n (

%)

WM GM HC CA PU PA AM AC VE

−0.02

0

0.02

0.04

****

**

***

** *

** **

*

Dic

e d

iffe

ren

ce

Improvement over foregr−backgr for multi−organ and latent label

IW Deconv g−kmeans g−DPmeans g−GMM g−DPGMM l−kmeans l−DPmeans

0.75

0.8

0.85

0.9

Dic

e

Left Parotid Gland

IW Deconv g−kmeans g−DPmeans g−GMM g−DPGMM l−kmeans l−DPmeans

0.6

0.65

0.7

0.75

0.8

0.85

Dic

e

Right Parotid Gland

IW Deconv g−kmeans g−DPmeans g−GMM g−DPGMM l−kmeans l−DPmeans

0.88

0.9

0.92

0.94

0.96

Dic

e

Left atrium

•  Quan&ta&ve  segmenta&on  results  for  three  applica&ons  •  Under-­‐segmenta&on  errors  significantly  higher  than  

over-­‐segmenta&on  errors   White  maDer  (WM),  Gray  maDer  (GM),  Hippocampus  (HC),  Caudate  (CA),    Putamen  (PU),  Pallidum  (PA),  Amygdala  (AM),  Accumbens  (AC),  Ventricles  (VE)  

First  column:  over-­‐segmenta&on  Second  column:  under-­‐segmenta&on  

Hypothesis  for  under-­‐segmenta1on  •  Under-­‐segmenta&on  is  caused  by  asymmetry    

in  foreground-­‐background  segmenta&on    •  Merging  all  surrounding  labels  into  background  creates  

a  new  meta-­‐label  that  dominates  the  vo&ng  process  •  Mul&-­‐organ  segmenta&on  of  brain  supports  hypothesis  

Latent  mul1-­‐label  model  of  the  background  •  Genera&ve  model  for  the  unsupervised  separa&on  of  the  

background  in  K  components  and  simultaneous  es&ma&on  of  K  •  Dirichlet  process  Gaussian  mixture  model  (DP-­‐GMM)  on  patches:  

•  Replace  background  label  with  mixture  component:    zix

2 {1, . . . ,K}S

i

(x) = z

ix

P = {Pix

: x 2 �, Si

(x) = b}

Si(x) = b

Red:  manual,  Yellow:  automa&c,  White:  voxel  of  interest    

Mul1-­‐atlas  segmenta1on  

Test  Image  

Registra&on  

Training  Data  

Images  

Segmenta&ons  

Propaga&on  

Test  Segmenta&on  

?

I = {I1, . . . , In}

S = {S1, . . . , Sn}

l 2 {1, . . . , ⌘}L

l(x) =nX

i=1

p(S(x) = l|Si) · p(I(x)|Ii)

p(S(x) = l|Si) =

⇢1 if Si(�i(x)) = l,

0 otherwise.

p(I(x)|Ii) / exp

��(I(x)� Ii(�i(x)))

2/2�

2�

ˆ

S(x) = argmax

lL

l(x)

L

f (x) > L

b(x)

S

Label  maps  specify  likelihood  of  each  label:  

Label  likelihood  term:  

Intensity  likelihood  term:    

Most  likely  label  yields  segmenta&on:  

Foreground-­‐Background  segmenta&on:  

Red:  manual,  Yellow:  automa&c    

Segmenta&on  of  paro&d  glands  in  CT  images  

Organ  

Background  

Merging  several  structures  to  one  background  label  

Left Parotid Right Parotid0

10

20

30

40

50

*

**

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

Left Atrium0

10

20

30

40

***

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

Paro&d  glands  in  head  &  neck  CT  Lef  atrium  in  heart  MRA  Nine  brain  structures  in  MRI  

Foreground-­‐background  segmenta&on  

Mul&-­‐organ  brain  segmenta&on  Illustra&on  of  mul&-­‐organ  and  foreground-­‐background  segmenta&on  

0 1 2 3 4 50.82

0.83

0.84

0.85

Dic

e

Merged Structures

Segmentation of Amygdala

Backgrd AM0

5

10

15

20

25

Vote

s

Frgrd−Bckgrd Voting

Lef  Atrium  

Left Atrium

0

5

10

15

20

25

30

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

Left Parotid Right Parotid

0

10

20

30

40

50

Ove

r− a

nd u

nder−

segm

enta

tion (

%)

Brain  

Paro&d  Glands  

U

O

Pix

|zix

⇠ N (µz

ix

⌃z

ix

)

zix

⇠ Cat(⇡)

⇡ ⇠ GEM(↵)

(µk

,⌃k

) ⇠ NW(�)

S: Manual ˆS: Automatic segmentation

S

S

WM GM HC AM Other0

5

10

15

20

25

Vo

tes

Multi−Organ Voting