MIT AI [email protected] Knowledge Based 3D Medical Image Segmentation Tina Kapur MIT Artificial...

MIT AI Lab [email protected]

Knowledge Based 3D Medical Image Segmentation

Tina KapurMIT Artificial Intelligence Laboratory

http://www.ai.mit.edu/~tkapur


Outline

• Goal of Segmentation

• Applications

• Why is segmentation difficult?

• My method for segmentation of MRI

• Future Work


The Goal of Segmentation


Applications of Segmentation

• Image Guided Surgery




• Surgical Simulation




• Surgical Simulation

• Neuroscience Studies

• Therapy Evaluation


Limitations of Manual Segmentation

• slow (up to 60 hours per scan)

• variable (up to 15% between experts)

[Warfield 95, Kaus98]


The Automatic Segmentation Challenge

An automated segmentation method needs to reconcile– Gray-level appearance of tissue– Characteristics of imaging modality– Geometry of anatomy


How to Segment? i.e. Issues in Segmentation of Anatomy



• Tissue Intensity Models



• Tissue Intensity Models– Parametric [Vannier]– Non-Parametric [Gerig]– Point distribution Models [Cootes]– Texture [Mumford]




• Imaging Modality Models




• Imaging Modality Models– MRI inhomogeneity [Wells]





• Anatomy Models: Shape, Geometric/Spatial





• Anatomy Models: Shape, Geometric/Spatial– PCA [Cootes and Taylor, Gerig, Duncan, Martin]– Landmark Based [Evans]– Atlas [Warfield]


Typical Pipeline for Segmentation of Brain MRI

pre-processing (noise removal)




intensity-based classification




post-processing (morphology/other)

intensity-based classification


Contributions of Thesis

• Developed an integrated Bayesian Segmentation Method for MRI that incorporates de-noising and global geometric knowledge using priors into EM-Segmentation

• Applied integrated Bayesian method to segmentation of Brain and Knee MRI.



• The Priors– de-noising: novel use of a Mean-Field

Approximation to a Gibbs random field in conjunction with EM-Segmentation (EM-MF)

– geometric: novel statistical description of global spatial relationships between structures, used as a spatially varying prior in EM-Segmentation


Background to My Work

• Expectation-Maximization Algorithm

• EM-Segmentation


Expectation-Maximization

• Relevant Literature:– [Dempster, Laird, Rubin 1977]– [Neal 1998]


Expectation-Maximization (what?)

• Search Algorithm

• for Parameters of a Model

• to Maximize Likelihood of Data

• Data: some observed, some unobserved


Expectation-Maximization (how?)

• Initial Guess of Model Parameters

• Re-estimate Model Parameters:– E Step: compute PDF for hidden variables,

given observations and current model parameters

– M Step: compute ML model parameters assuming pdf for hidden variables is correct


• Notation– Observed Variables:– Hidden Variables :– Model Parameters:

Expectation-Maximization (how exactly?)

HV


• Initial Guess:

• Successive Estimation of

– E Step:

– M Step:

Expectation-Maximization (how exactly?)

)0(,..., )1()0(

),|()(~ )1()( tt VHPHP

)]'|,([logmaxarg )(~'

)(

VHPE tP

t


Expectation-Maximization

• Summary/Intuition:– If we had complete data, maximize likelihood– Since some data is missing, approximate

likelihood with its expectation– Converges to local maximum of likelihood


EM-Segmentation [Wells 1994]

• Observed Signal is modeled as a product of the true signal and a corrupting gain field due to the imaging equipment

• Expectation-Maximization is used on log-transformed observations for iterative estimation of – tissue classification – corrupting bias field (inhomogeneity correction)


M-Step

E-Step



Estimate intensity correctionusing residuals based on current posteriors.

Compute tissue posteriors using current intensity correction.

M-Step

E-Step



• Observed Variables– log transformed intensities in image

• Hidden Variables– indicator variables for classification

• Model Parameters – the slowly varying corrupting bias field

( refer to variables at voxel s in image)

Y

W

sss WY ,,



• Initial Guess:


– E Step:

– M Step:

)0(,..., )1()0(

),|()(~ )1()( tt YWPWP

)],|'([logmaxarg )(~'

)( YWPE tP

t



Situating My Work

• Prior in EM-Segmentation:– Independent and Spatially Stationary

• My contribution is addition of two priors:– a spatially stationary Gibbs prior to model local

interactions between neighbors (thermal noise)– spatially varying prior to model global

relationships between geometry of structures


The Gibbs Prior

• Gibbs Random Field (GRF)

– natural way to model piecewise homogeneous phenomena

– used in image restoration [Geman&Geman 84]

– Probability Model on a lattice– Partially Relaxes independence assumption to

allow interactions between neighbors


EM-MF Segmentation: EM + Gibbs Prior

• We model tissue classification W as a Gibbs random field:


• We model tissue classification W as a Gibbs random field:

Model Gibbs with associatedenergy

parameter etemperatur

))'(1

(exp

))(1

(exp1

)(

'

P

WP

P

E

T

WET

Z

WETZ

WP

EM-MF Segmentation: Gibbs Prior on Classification


• To fully specify the Gibbs model:– define neighborhood system as a first

order neighborhood system i.e. 6 closest voxels– use to define

N

PE

stat

ss

Tr

s Nr

Tsp

pf

J

WfWJWWEs

log

matrixn interactio class-tissue

21

)(

N

EM-MF Segmentation: Gibbs Prior on Classification


EM-MF Segmentation: Gibbs form of Posterior

• Gibbs prior and Gaussian Measurement Models lead to Gibbs form for Posterior:

))(1

(exp1

),|( WETZ

YWP t


• Gibbs prior and Gaussian Measurement Models lead to Gibbs form for Posterior:

),|(log

21

),|(log)()(

))'(1

exp( where

)(

)(

'

tsissisi

ss

Tr

s Nr

Ts

tp

W

eWYpfg

WgJWW

WYPWEWE

WET

Z

s

))(1

(exp1

),|( WETZ

YWP t

EM-MF Segmentation: Gibbs form of Posterior


EM-MF Segmentation

• For E-Step: Need values for ),|( YWP


EM-MF Segmentation

• For E-Step: Need values for

• Cannot compute directly from Gibbs form

),|( YWP


EM-MF Segmentation

• For E-Step: Need values for


• Note

),|( YWP

],|[),|( YWEYWP


EM-MF Segmentation• For E-Step: Need values for


• Note

• Can approximate – Mean-Field Approximation to GRF

),|( YWP

],|[),|( YWEYWP

WYWE ],|[ W


Mean-Field Approximation• Deterministic Approximation to GRF [Parisi84]

– the mean/expected value of a GRF is obtained as a solution to a set of consistency equations

• Update Equation is obtained using derivative of partition function with respect to the external field g. [Elfadel 93]

• Used in image reconstruction [Geiger, Yuille, Girosi 91]


Mean-Field Approximation to Posterior GRF

M

i Ns

M

ksikski

Ns

M

ksiksik

si

s

s

gWJ

gWJW

1' ' 1'''

' 1'

)(exp

)(exp

Intuition:•denominator is normalizer•numerator captures:

•effect of labels at neighbors•measurement at voxel itself


Summary of EM-MF Segmentation

• Modeled piecewise homogeneity of tissue using a Gibbs prior on classification

• Lead to Gibbs form for Posteriors

• Posterior Probabilities in E-Step are approximated as a Mean-Field solution


EM-MF Results

• Application: Brain MRI – white matter, gray matter, fluid/air, skin/scalp

• Results

• Comparison with Manual Segmentation


Some ResultsEM EM-MF


More Results

Noisy MRI EM Segmentation EM-MF Segmentation


Posterior Probabilities (EM)

Whitematter

Graymatter


Posterior Probabilities (EM-MF)

Whitematter

Graymatter


Results

Noise =10 Noise =20

EM EMMF % + EM EMMF %+1 90% 92% 2% 84% 91% 7%2 86% 91% 5% 80% 87% 7%3 87% 90% 3% 82% 89% 7%

4 88% 92% 4% 83% 90% 7%

5 87% 91% 4% 81% 89% 8%


Modeling Global Geometric Relationships between Structures


• Relative Geometry Models

• Motivate Using Knee MRI

• Brain MRI Example

Modeling Global Geometric Relationships between Structures


Segmented Knee MRI

Femur

Tibia

Femoral Cartilage

Tibial Cartilage

MERL, SPL, MIT, CMUSurgical Simulation(Sarah Gibson, PI)


Motivation

• Primary Structures– image well– easy to segment

• Secondary Structures– image poorly– relative to primary

Tibial Cartilage

Femoral Cartilage

Tibia

Femur


Relative Geometric Prior Approach

• Select primary/secondary structures• Measure geometric relation between primary

and secondary structures from training data • Given novel image

– segment primary structures

– use geometric relation as prior on secondary structure in EM-MF Segmentation


Segment Primary Structures: Femur, Tibia

Seed Region Growing Boundary Localization


Status

• Have Bone

• Want Cartilage


Measure Geometric Relationship between Primary and Secondary

Structures Femur

Tibia

Femoral Cartilage

Tibial Cartilage

• Using primitives such as– distances between surfaces– local normals of primary structures– local curvature of primary structures– etc.


Femur

Tibia

Femoral Cartilage

Tibial Cartilage

Measure Geometric Relationship between Primary and Secondary

Structures

Ze)P(CartilagCartilage)x|nP(

)Bone|CartilageP(x

pointclosest at (femur) bone tonormal n

(femur) boneon point closest todistance

sss

s

s

s

,


Estimate of Cartilage)x|nP( sss ,

Cartilage) Fem.x|nP( sss , Cartilage) Tib.x|nP( sss ,


Status

• Have Bone

• Have spatial relation between Bone and Cartilage

• Need Cartilage


Use Relative Geometric Prior in EM Segmentation

Replace stationary prior with relative geometric prior:

)Bone|CartilageP(xe)P(Cartilag i


Results: Segmentation ofFemoral & Tibial Cartilage

MRI Image Model-BasedSegmentation

Manual Segmentation


Relative Geometric Priors for Brain Tissue

• Prior Estimation– Select primary structures (boundary of skin,

ventricles)

– Estimate

• Using Prior in Segmentation– Segment primary structures: skin, ventricles

– Use as geometric prior

Tissue)Brain x|P( s21 ,

)|TissueBrain xP( 21s ,


White Matter Gray Matter

Estimate

distance to Ventricles dist

ance

to S

kin

Tissue)Brain x|P( s21 ,


Resultant Segmentation

MRI EM SegmentationEM-MF

withGeometric Prior


Posterior Probabilities

Gray Matter

White Matter

EM-MF+Geometric PriorEM


In Summary

• Incorporated robustness to thermal noise by using Mean-Field Approximation to Gibbs model in conjunction with EM Segmentation. Applied to Brain MRI.

• Introduced Relative-Geometry Models and applied to Brain and Knee MRI.


Future Work

• Further development of Relative-Geometry Models:– Automatic selection of primary/secondary

structures – Additional primitives for Spatial Relationships


STOP


d1 (distance to Ventricles)

d2 (

dist

ance

to S

kin)

White Matter

Grey Matter

FluidAir

Left Caudate

Right Caudate

Class Conditional Density


Unified Bayesian Segmentation Method

Simultaneous•noise reduction•intensity-based classification•use of geometric information

for segmentation.


Rest of Talk:

1.The Unified Segmentation Method

2.Two Priors

3.Results on Brain, Knee Segmentation

4.Conclusions


The Gibbs Prior• To fully specify the Gibbs model:

– define neighborhood system as a first order neighborhood system i.e. 6 closest voxels

– use to define

N

PE

stat

ss

Tr

s Nr

Tsp

pf

J

WfWJWWEs

log

matrixn interactio class-tissue

21

)(

N


Components of EM Framework

• Measurement models for tissue

• Prior models for tissue

• Model for bias field– piecewise smooth


Addition of Two Priors

• Gibbs prior on tissue appearance– models tissue as piecewise constant

• Geometric Prior to encode spatial relations – gray matter is outside ventricles and inside

skull


Gibbs Model


Gibbs Model

• probability model on a lattice

• independence assumption is partially relaxed

• spatial range of interaction is local neighborhood


Gibbs Model

• probability model on a lattice

• independence assumption is partially relaxed

• spatial range of interaction is local neighborhood

Mean-Field ApproximationApproximates neighboring random variables with theirmean values:

- algebraic and computational simplicity



MRI+Noise EM SegmentationEM-MF

withGeometric Prior


Proposed MRI Segmentation Method

• Bayesian Statistical Classification Scheme that uses Expectation-Maximization

• Replaces pipeline with Priors on intensity and geometry


Proposed MRI Segmentation Method

• Previous Work [Wells 1994, 1996]– derived as a special case– spatially stationary, independent priors– piecewise smooth inhomogeneity model

• This Work:– locally interacting prior for intensity– spatially varying prior for geometry


Next

• Background on Expectation-Maximization


3. Specify Prior on Classification as a Gibbs Random Field.

4. Gibbs Prior + Gaussian Measurement Model imply is also a GRF.

5. Approximate using Mean-Field Solution for the GRF.

),|()(~ )1()( tt YWPWP Computation of )(WP

)(,| tYW

),|( )(tYWP


Recap: 1. Bayes’ Rule to rewrite …

2. Gaussian Measurement Models

3. Gibbs form of Prior

4. Gibbs form of Posterior

5. Mean-Field Approximation to Gibbs form

),|()(~ )1()( tt YWPWP Computation of


The Gibbs Prior• Tissue-class interaction Matrix

• Stationary Prior

data gin trainin occurs #data gin trainin next to occurs#

logk

lkJ kl

data gin trainin voxels#data gin trainin occurs#

loglogi

pf statii


The Mean-Field Solution• Gibbs models can be solved using

– [Metropolis 1953], [Geman and Geman 1984], [Besag 1986] etc.

• We use Mean-Field Approximation to estimate the expected value of the posterior GRF as a solution to a set of consistency equations.


The Mean-Field Approximation• Deterministic Approximation

• Update Equation is obtained using derivative of partition function with respect to the external field g.


The Mean-Field Solution

M

i Ns

M

ksikski

Ns

M

ksiksik

si

s

s

gWJ

gWJW

1' ' 1'''

' 1'

)(exp

)(exp

Intuition:•denominator is normalizer•numerator captures:

•effect of labels at neighbors•measurement at voxel itself


• Initial Guess:


– E Step: Estimate as• Mean-Field Solution to a Gibbs Random Field

– M Step: Compute same as [Wells 1996]

EM-MF Summary

)0(,..., )1()0(

),|( )1( tYWP

)(t

W

MIT AI [email protected] Knowledge Based 3D Medical Image Segmentation Tina Kapur MIT Artificial...

Documents

Transcript of MIT AI [email protected] Knowledge Based 3D Medical Image Segmentation Tina Kapur MIT Artificial...