Levelset Tutorial

8/9/2019 Levelset Tutorial

1/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes

Nikos Paragioshttp://cermics.enpc.fr/~paragios

CERTISCERTISEcole Nationale des Ponts et ChausseesEcole Nationale des Ponts et Chaussees

Paris !ranceParis !rance

Level Set Methods in Medical ImageAnalysis: Segmentation


2/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes


"tlantis Research #roup"tlantis Research #roupEcole Nationale des Ponts et ChausseesEcole Nationale des Ponts et ChausseesParis !ranceParis !rance

Stanle$ %sher

http://math.ucla.edu/~s&o

'epartment of (athematics'epartment of (athematics)ni*ersit$ of California +os "ngeles)ni*ersit$ of California +os "ngeles

)S")S"

,ttp://cermics.enpc.fr/~paragios/-ook/-ook.html


3/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes

Outline

Introduction/(oti*ation

%n the Propagation of Cur*es The snake model

The le*el set method asic 'eri*ation algorithms

oundar$dri*en and Regiondri*en model free segmentation

The +e*el Set (ethod as a 'irect %ptimi0ation Space (ultiphase (otion Regiondri*en model free image segmentation

1no2ledge-ased %-&ect E3traction

Shape Registration

'iscussion


4/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes

Motivation

Image Segmentation and image registration are core componentsof medical imaging

4554

The 2ord 6Segmentation7 appears 89 times at (ICC"I54 program

The 2ord 6Registration7 appears 44 times at (ICC"I54 program

4558:

The 2ord 6Segmentation7 appears 9; times at (ICC"I58 program ~ 4


5/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes

Overview of Segmentation Techniques

oundar$dri*en Edge 'etectors @model freeA

"cti*e Contours/snakes @model free B kno2ledge-asedA

"cti*e Shape (odels @kno2ledge-asedA

Regiondri*en 'eforma-le templates @kno2ledge-asedA

Statistical/clustering techniues @model free B kno2ledge-asedA

(R!-ased techniues @model freeA

"cti*e "ppearance (odels @kno2ledge-asedA

oundar$ B Regiondri*en "cti*e Contours @model free B kno2ledge-asedA

#raph-ased Techniues @model freeA

+e*el Set (ethods @model free B kno2ledge-asedA


6/92

Ce

ntre

dEeignemente

tde

Recherche

enTechnologies

deL

Information

etSystemes

On the propagation of Curves


7/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

On the ropagation of Curves

Snake (odel @>D;A F1assGitkinTer0opoulosH

Planar parameteri0ed cur*e C:RR3R

" cost function defined along that cur*e

The internal termstands for regularit$/smoothness along the cur*eand has t2o components @resisting to stretching and -endingA

The image term guides the acti*e contour to2ards the desired image

properties @strong gradientsA The e3ternal termcan -e used to account for userdefined

constraints or prior kno2ledge on the structure to -e reco*ered

The lo2est potential of such a cost function refers to an euili-riumof these terms


8/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

Active Contour Components

The internal termJ

The first order deri*ati*e makes the snake -eha*e as a mem-rane

The second order deri*ati*e makes the snake act like a thin plate

The image termJ

Can guide the snake to Isophotes edges

and terminations

Numerous Pro*isionsJ: -alloon models regionsnakes etcJ


9/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

Optimi!ing Active Contours

Taking the Euler+agrange euations:

That are used to update the position of an initial cur*e to2ardsthe desired image properties

Initial the cur*e using a certain num-er of control points as 2ell as aset of -asic functions

)pdate the positions of the control points -$ sol*ing the a-o*eeuation

Reparameteri0e the e*ol*ing contour and continue the process untilcon*ergence of the processJ


10/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

ros"Cons of such an approach

Pros +o2 comple3it$

Eas$ to introduce prior kno2ledge

Can account for open as 2ell as closed structures

" 2ell esta-lished techniue numerous pu-lications it 2orks

)ser Interacti*it$ 'emetri Ter0opoulos is a *er$ good friend

Cons Selection on the parameter space and the sampling rule affects the

final segmentation result Estimation of the internal geometric properties of the cur*e in

particular higher order deri*ati*es

Kuite sensiti*e to the initial conditions

Changes of topolog$ @some efforts 2ere done to address the pro-lemA


11/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

Level Set: The #asic $erivation


12/92

Ce

ntre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes

The Level Set Method

%sherSethian @>D;A Earlier: 'er*ieu3 Thomassett @>D;D >D5A

Introduced in the area of fluid d$namics

Lision and image segmentation CasellesCattecoll'i-os @>DD4A

(alladiSethianLermuri @>DD9A

+e*el Set (ilestones

!augeraskeri*en @>DDA stereo reconstruction Paragios'eriche @>DDA acti*e regions and grouping

ChanLese @>DDDA mumfordshah *ariant

+e*enton#rimson!augerasetal @4555A shape priors

Mhao!edkie2%sher @455>A computer graphics


13/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes


+et us consider in the most general case the follo2ing form ofcur*e propagation:

"ddressing the pro-lem in a higher dimensionJ

The le*el set method represents the cur*e in the form of animplicit surface:

That is deri*ed from the initial contour according

to the follo2ing condition:


14/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes


Construction of the implicit function

"nd taking the deri*ati*e 2ith respect to time @using the chain ruleA

"nd 2e are '%NEJ

@>A


15/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSystemes


+et us consider the arclength @cA parameteri0ation of the cur*e

then taking the directional deri*ati*e of in that direction 2e 2ill o-ser*e no change:

leading to the conclusion that the is orthonormal to C 2here

the follo2ing e3pression for the normal *ector

Em-edding the e3pression of the normal *ector to:

the follo2ing flo2 for the implicit function is reco*ered:

@4A


16/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSys

temes

Level Set Method %the #asic derivation&

Ghere a connection -et2een the cur*e propagation flo2 and theflo2 deforming the implicit function 2as esta-lished

#i*en an initial contour an implicit function is defined anddeformed at each pi3el according to the euation @4A 2here the0erole*el set corresponds to the actual position of the cur*e at agi*en frame

Euclidean distance transforms are used in most of the cases asem-edding function


17/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSys

temes

Overview of the Method

The le*el set flo2 can -e re2ritten in the follo2ing form

2here , is kno2n to -e the ,amiltonian. Numerical appro3imationsis then done according to the form of the ,amiltonian

'etermine the initial implicit function @distance transformA E*ol*e it locall$ according to the le*el set flo2

Reco*er the 0erole*el set isosurface @cur*e positionA

Reinitiali0e the implicit function and #o to step @>A of the loop

Computationall$ e3pensi*e

%pen Kuestions: reinitiali0ationJand numerical appro3imations


18/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSys

temes

Implementation $etails'


19/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

deL

Information

etSys

temes

The normal to the cur*e/surface can -e determined directl$ fromthe le*el set function:

The cur*ature can also -e reco*ered from the implicit function -$taking the second order deri*ati*e at the arc length

Ghere 2e o-ser*e no *ariation since the implicit function hasconstant 60ero7 *alues and gi*en that as 2ellas one can easil$ pro*e that:

That can also -e e3tended to higher dimensions

Level Set Method and Internal Curve roperties


20/92

Centre

dEeignemente

tde

Recherche

enT

echnologies

de

LInformation

etSys

temes

(inimi0e the Euclidean length of acur*e/surface:

The corresponding le*el set *ariant 2ith a

distance transform as an implicit function:

Things -ecome little -it more complicated at8' @#aussian Cur*atureA

Results are courtes$ Prof. . Sethian

@erkele$A O #. ,ermosillo @INRI"A

()amples: Mean"*aussian Curvature +low


21/92

Centre

dEeignementetde

Recherche

enT

echnologies

de

LInformation

etSys

temes

+rom theory to ractice %,arrow -and&.Chop:/01 Adalsteinsson2Sethian:/34

Central idea: 2e are interested on the motion of the 0erole*elset and not for the motion of each isophote of the surface

E3tract the latest position

'efine a -and 2ithin a certain distance

)pdate the le*el set function Check ne2 position 2ith respect

the limits of the -and

)pdate the position of the -and

regularl$ and reinitiali0e the implicit function

Significant decrease on the computational comple3it$ inparticular 2hen implemented efficientl$ and can account for an$t$pe of motion flo2s


22/92

Centre

dEeignementetde

Recherche

enT

echnologies

de

LInformation

etSys

temes

,arrow -and %the #asic derivation&

Results are courtes$: R. 'eriche


23/92

Centre

dEeignementetde

Recherche

enT

echnologies

de

LInformation

etSys

temes

5andling the $istance +unction

The distance function has to -e freuentl$ reinitiali0edJ

E3traction of the cur*e position O reinitiali0ation:

)sing the marching cu-es one can reco*er the current position of the cur*e

set it to 0ero and then reinitiali0e the implicit function: the orgefors

approach the !ast (arching method e3plicit estimation of the distancefor all image pi3elsJ

Preser*ing the cur*e position and refinement of the e3isting function

@Susmansmerekaosher:D9A

(odification on the le*el set flo2 such that the distance transformpropert$ is preser*ed @gomesfaugeras:55A

E3tend the speed of the 0ero le*el set to all isophotes rather complicated

approach 2ith limited added *alue


24/92

Centre

dEeignementetde

Recherche

enT

echnologies

de

LInformation

etSys

temes

+rom theory to ractice %+ast Marching&.Tsitsi6lis:/01Sethian:/34

Central idea: 6mo*e7 the cur*e one pi3el in a progressi*e manneraccording to the speed function 2hile preser*ing the nature ofthe implicit function

Consider the stationar$ euation

Such an euation can -e reco*ered for all flo2s 2herethe speed function has one sign @either positi*e or negati*eApropagation takes place at one direction

If T@3$A is the time 2hen the implicit function reaches @3$A:


25/92

Centre

dEeignementetde

Recherche

enT

echnologies

de

LInformation

etSys

temes

+ast Marching %continued&

Consider the stationar$ euation in its discrete form:

"nd using the assumption

that the surface propaga tes in one direction the so

lution can -e o-tained -$

out2ards propagation from

the smallest T *alueJ

acti*e pi3els the cur*e has alread$ reached them

ali*e pi3els the cur*e could reach them at the ne3t stage

far a2a$ pi3els the cur*e cannot reach them at this stage


26/92

Centre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

+ast Marching %continued&

INITI"+ STEP

Initiali0e for the all pi3els of the front @acti*eA their firstorder neigh-ors ali*eand the rest far a2a$

!or the first order neigh-ors

estimate the arri*al time according to: Ghile for the rest the crossing time is set to infinit$

PR%P"#"TI%N STEP Select the pi3el 2ith the lo2est arri*al time from the ali*eones

Change his la-el from ali*eto acti*e and for his first order neigh-ors: If the$ are ali*e update their T *alue according to

If the$ are far a2a$ estimate the arri*al time according to:


27/92

Centre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

+ast Marching ros"Cons1 Some 7esults

!ast approach for a le*el setimplementation

Ler$ efficient techniue for resetting the

em-edding function to -e distance

transform

Single directional flo2s great importance

on initial placement of the contours

"-sence of cur*ature related terms or

terms that depend on the geometric

properties of the cur*eJ

Results are courtes$: . Sethian R. (alladi

T. 'eschamps +. Cohen


28/92

Centre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

Level Sets in imaging and vision'the edge2driven case


29/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

(migration from +luid $ynamics to 8ision

@CasellesCateColl'i-os:D8(alladiSethianLemuri:D9A ha*eproposed geometric flo2s to -oundar$ e3traction

Ghere g@QA is a function that accounts for strong image

gradients

"nd the other terms are application specificJthat either

e3pand or shrink constantl$ the initial cur*e

'istance transforms ha*e -een used as em-edding functions

(alladiSethianLemuri:D9


30/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*eodesic Active Contours.Caselles29immel2Sapiro:/31 9ichenassamy29umar2etal/34

Connection -et2een le*el set methods and snake dri*enoptimi0ation

The geodesic acti*e contour consists of a simplified snake model2ithout second order smoothness

That can -e 2ritten in a more general form as

Ghere the image metric has -een replaced 2ith a monotonicall$decreasing function:


31/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*eodesic Active Contours.Caselles29immel2Sapiro:/31 9ichenassamy29umar2etal/34

+eading to the follo2ing more general frame2orkJ

%ne can assume that smoothness as 2ell as image terms areeuall$ important and 2ith some 6-asic math7

That seeks a minimal length geodesic cur*e attracted -$ thedesired image propertiesJ


32/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*eodesic Active Contours

That 2hen minimi0ed leads to the follo2ing geometric flo2:

'atadri*en constrained -$ the cur*ature force #radient dri*en term that ad&usts the position of the contour 2hen

close to the real 5-&ect -oundariesJ

$ em-edding this flo2 to a le*el set frame2ork and using a

distance transform as implicit function


33/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*eodesic Active Contours'

That has an e3tra term 2hen

compared 2ith the flo2 proposed

-$ (alladiSethianLemuri.

Single directional flo2Jreuires

the initial contour to either

enclose the o-&ect or to -e

completel$ inside...

Results are courtes$: R. 'eriche


34/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*radient 8ector +low *eometric Contours.paragios2mellina2ramesh:;4

Initial conditions are an issue at the acti*e contours since the$are propagated mainl$ at one direction

Region terms @later introducedA isa mean to o*ercome this limitationJ

an alternati*e is someho2 to e3tendthe -oundar$dri*en speed function to account for directionalit$thus reco*ering a field @u*A

%ne can estimate this field close to the o-&ect -oundariesJ2here

The image gradient at the -oundaries is tangent to the cur*e

Ghile the in2ard normal normal points to2ards the o-&ect -oundaries


35/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSys

temes

*radient 8ector +low *eometric Contours.paragios2mellina2ramesh:;4

+et @fA -e a continuous edge detector 2ith *alues close to > at thepresence of noise and 5 else2hereJ

The flo2 can -e determined in areas 2ith important -oundar$information @Important fA

"nd areas 2here there changes on f #radient@fA

Ghile else2here reco*ering such a field is not possi-le and the onl$2a$ to -e done is through diffusion

This can -e done through an appro3imation of image gradient at theedges and diffusion of this information for the rest of the imageplane


36/92

C

entre

dEeignementetde

Recherche

en

Technologies

de

LInformation

etSystemes

*radient 8ector +low *eometric Contours

This flo2 can -e used 2ithin a geometric flo2 to2ards imagesegmentationJ

The direction of the propagation should -e the same 2ith the oneproposed -$ the reco*ered flo2 therefore one can penali0e theorientation -et2een these t2o *ectors.

That is integrated 2ithin the classical

#eodesic acti*e contour euation and is

implemented using the le*el set function using the "dditi*e %perator Splitting

The inner product -et2een the cur*e

normal and the *ector field guides the cur*e propagation


37/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Additive Operator Splitting.


38/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Additive Operator Splitting %&

%r in a semiimplicit one

That refers to a triagonal s$stem of euations and can -e done

using the Thomas algorithmJat %@NA and has to -e done onceJ


39/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Some Comparison %&


40/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Level Sets in imaging and vision'the region2driven case


41/92


42/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

The Mumford2Shah framewor6.chan2vese://1 ye!!i2tsai2wills6y2//4

Taking the deri*ati*es 2ith respect to piece2ise constants itstraightfor2ard to sho2 that their optimal *alue corresponds tothe means 2ithin each region:

Ghile taking the deri*ati*es 2ith respect and using the stokestheorem the follo2ing flo2 is reco*ered for the e*olution of thecur*e:

"n adapti*e @directional/magnitudeA2ise -alloon force

" smoothness force aims at minimi0ing the length of the partition That can -e implemented in a straightfor2ard manner 2ithin the

le*el set approach


43/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

The Mumford2Shah framewor6 ?Criticism @ 7esults

"ccount for multiple classes Kuite simplistic model uite often the means are not a good

indicator for the region statistics "-sence of use on the edges -oundar$ information


44/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

*eodesic Active 7egions.paragios2deriche:/=4

Introduce a frame partition paradigm 2ithin the le*el set spacethat can account for -oundar$ and glo-al regiondri*eninformation

1E "SS)(PTI%NS

%ptimi0e the position and the geometric form of the cur*e -$measuring information along that cur*e and 2ithin the regions thatcompose the image partition defined -$ the cur*e:

@input imageA @-oundar$A @regionA


45/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

*eodesic Active 7egions

Ge assume that prior kno2ledge on the positions of the o-&ectsto -e reco*ered is a*aila-le as 2ell as on the e3pectedintensit$ properties of the o-&ect and the -ackground


46/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes


Such a cost function consists of: The geodesic acti*e contour

" regiondri*en partition module that aims at separating theintensities properties of the t2o classes @see later analog$ 2ith the(umfordShahA

"nd can -e minimi0ed using a gradient descent method leading to:

Ghich can -e implemented using the le*el set method as follo2sJ


47/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes



48/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Some 7esults'


49/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

'7(MI,$(7'


50/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Level Set @ *eometric +lows

Ghile e*ol*ing mo*ing interfaces 2ith the le*el set method isuite attracting still it has the limitation of -eing a staticapproach

The motion euations are deri*ed someho2

The le*el set is used onl$ as an implementation toolJ

That is eui*alent 2ith sa$ing that the pro-lem has -een someho2

alread$ sol*edJsince there is not direct connection -et2een theapproach and the le*el set methodolog$


51/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Level Set: Optimi!ation space


52/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Level Set $ictionary

+et us consider distance transformsas em-edding function

Then follo2ing ideas introduced in Fe*ansgariep$:D?H one can introduce the 'irac distri-ution


53/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Level Set $ictionary

)sing the 'irac function and integrating 2ithin the image domainone can estimate the length of the cur*e:

Ghile integrating the ,ea*iside 'istri-ution 2ithin the image

domain

Such o-ser*ations can -e used to define regional partitionmodules as follo2s according to some descriptors

That can -e optimi0ed 2ith respect to the le*el set function@implicitl$ 2ith respect to a cur*e positionA


54/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Level Set Optimi!ation

"nd gi*en that :

"n adapti*e @directional O magnitude 2iseA flo2 is reco*ered forthe propagation of an initial surface to2ards a partition that isoptimal according to the regional descriptorsJ

The same idea can -e used to introduce contourdri*en termsJ


55/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Level Set Optimi!ation

and optimi0e them directl$ on the le*el set space

Cur*edri*en terms:

#lo-al regiondri*en terms:

"ccording to some image metricsJdefined along the cur*e and2ithin the regions o-tained through the image partition accordingto the position of the cur*e that can -e multicomponent -ut isrepresenting onl$ one class


56/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multiphase Motion .!hao2chan2merinman2osher:/4

)p to no2 statistics and image information ha*e -een used topartition image into t2o classes

%ften 2e need more than o-&ect/-ackground separation andtherefore the case of multiphase motion is to -e consideredJ

N o-&ects/cur*es represented -$ N le*el set functions

,o2 to deal 2ith occlusions

one image pi3el cannot -e

assigned to more than one cur*eJ

,o2 to constrain the solution

such that the o-tained partition

consists of all image data


57/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multi2hase Motion %continued&

!or each class -oundar$ smoothness as 2ell as region componentscan -e considered

Su-&ect to the constraint at each pi3el:

a hard and local constraint difficult to -e imposed that could -e

replaced 2ith a more con*enient

That can -e optimi0ed through +agrange multipliers methodJ


58/92

C

entre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multiphase Motion @ Mumford2Shah.samson2au#ert2#lanc2feraud://4

Image Segmentation and Signal Reconstruction @direct application ofthe @0haochanmerinmanosher:D?A 2ithin the (umford Shah

formulationJA

Separate the image into regions 2ith consistent intensit$ properties

Reco*er a #aussian distri-ution that e3presses the intensit$properties of each class or force the intensit$ properties of eachclass to follo2 some predefined image characteristics

That 2hen optimi0ed leads to a set of euations that deformingsimultaneousl$ the initial cur*es according to:


59/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multiphase Motion @ Mumford2Shah.samson2au#ert2#lanc2feraud://4


60/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multi2hase Motion

PR%S Taking the le*el set method to another le*el

'ealing 2ith multiple @multicomponentA o-&ects and multiple tasks

Introducing interactions -et2een shape structures that e*ol*e inparallel

C%NS

Computationall$ e3pensi*e

'ifficult to guarantee con*ergence

Numericall$ unsta-le O hard to implement Prior kno2ledge reuired on the num-er of classes and in some cases

on their propertiesJ

P"RTI"+ S%+)TI%N: The multiphase ChanLese model


61/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Introduce classification according to a com-ination of all le*elsets at a gi*en pi3el

+ELE+ SET 'ICTI%N"R

Class >:

Class 4: Class 8:

Class 9:

"nd therefore -$ taking these products one can define a modified *ersion

of the mumfordshah approach to account 2ith four classes 2hile usingt2o le*el set functionsJ

Multi2hase Motion.vese2chan:>4


62/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multi2hase Motion


63/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Multi2hase Motion with more advanced data2driven terms

The assumption of piece2ise constant is rather 2eak in particularin medical imagingJ

Se*eral authors ha*e proposed more ad*anced statisticalformulations that are reco*ered 6on the fl$7 to determine thestatistics of each class

The case of nonparametric appro3imations of the histogram2ithin each region is a promising direction


64/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased O#Bect ()traction


65/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased O#Bect ()traction

%-&ecti*e:reco*er from the image a structure

of a particular kno2n to some e3tend

geometric form

(ethodolog$

Consider a set of training e3amples Register these e3amples to a common pose

Construct a compact model that e3presses the

*aria-ilit$ of the training set

#i*en a ne2 image reco*er the area 2here the

underl$ing o-&ect looks like that one learnt "d*antages of doing that on the +S space:

Preser*e the implicit geometr$

"ccount 2ith multicomponent o-&ectsJ

J all 2onderful staff $ou can do 2ith the +S


66/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.leventon2faugeras2grimson2etal:4

Concept: "lternate -et2een segmentationO imposing prior kno2ledge

+earn a #aussian distri-ution of the

shape to -e reco*ered from a training

set directl$ at the space of implicit functions The elements of the training set are registered

" principal component anal$sis is use to reco*er

the co*ariance matri3 of pro-a-ilit$ densit$ function of this set

"+TERN"TE

E*ol*e a let set function according to the geodesic acti*e contour #i*en its current form deform it locall$ using a ("P criterion so it fits

-etter 2ith the prior distri-ution

)ntil con*ergenceJ


67/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.leventon2faugeras2grimson2etal:4

+imitations: 'ata dri*en O prior term are decoupled

uilding densit$ functions on high dimensional spaces is an ill posedpro-lem

'ealing 2ith scale and pose *ariations @the$ are not e3plicitel$addressedA


68/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.chen2etal:;4

Concept le*el:

)se an a*erage model as prior in its implicit function

!or a gi*en cur*e find the transformation that pro&ects it closer to the0erole*el set of the implicit representation of the prior

!or a gi*en transformation e*ol*e the cur*e locall$ to2ards -etterfitting 2ith the priorJ

Couple prior 2ith the image dri*en term in a direct formJ

Issues to -e addressed: (odel is *er$ simplistic @a*erage shapeA opposite to the le*entons case

2here it 2as too much complicatedJ

Estimation of the pro&ection -et2een the cur*e and the model space is

trick$Jnot enough supportJdata term can -e impro*edJ


69/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.chen2etal:;4


70/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.tsai2ye!!i2etal:;4

"t a concept le*el prior kno2ledge is modeled through a #aussiandistri-ution on the space of distance functions -$ performing asingular *alue decomposition on the set of registered training set

The mumfordshah frame2ork determined at space of the model

is used to segment o-&ects according to *arious datadri*en terms

The parameters of the pro&ection are reco*ered at the same time2ith the segentation resultJ

" more con*enient approach than the one of +e*entonetal

Ghich suffers from not comparing directl$ the structure that isreco*ered 2ith the modelJ


71/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

9nowledge2#ased Segmentation.paragios2rousson:>4

Prior is imposed -$ direct comparison -et2een the model ande*ol*ing contour modulo a similarit$ transformationJ

The model consists of a stochastic le*el set 2ith t2o components " distance map that refers to the a*erage model

"nd a confidence map that dictates the accurac$ of the model

%-&ecti*e: Reco*er a le*el set that pi3el2ise looks like the prior

modulo some transformation


72/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Model Construction

!rom a training set reco*er the most representati*e modelQ

If 2e assume N samples on the training set then the distri-utionthat e3presses at a gi*en point most of these samples is the onereco*ered through ("P

Ghere at a gi*en pi3el 2e reco*er the mean and the *ariance that-est descri-es the training set composed of implicit functions atthis point 2here the mean corresponds to the a*erage *alue

Constraints on the *ariance to -e locall$ smooth is a naturalassumption


73/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Model Construction %continued&

The calculus of *ariations can lead to the estimation of the meanand *ariance @confidence measureA of the model at et each pi3el

,o2e*er the resulting model 2ill not -e an implicit function in thesense of distance transform @a*eraging distance transforms

doesnt necessar$ produce oneA

%ne can seek for a solution of the pre*iousl$ defined o-&ecti*efunction su-&ect to the constraint the 6means7 field forms adistance transform using +agrange multipliersJ

"n alternati*e is to consider the process in repeated steps 2herefirst a solution that fits the data is reco*ered and then ispro&ected to the space of distance functionsJ


74/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Imposing the %Static& rior

'efine/reco*er a morphing function 6"7 that createscorrespondence -et2een the model and the prior

In the a-sence of scale *ariations and in the case of glo-almorphing functions one can compare the e*ol*ing contour 2ith themodel according to

That modulo the morphing function 2ill e*ol*e the contours to2ardsa -etter fit 2ith the model

%ne can pro*e that scale *ariations introduce a multiplicati*e factorand the$ ha*e to -e e3plicitl$ taken into account


75/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Static rior %continued&

Ghere the unkno2ns are the morphing function and the positionof the le*el set

Calculus of *ariations 2ith respect to the position of theinterface are straightfor2ard:

The second term is a constant inflation term aims at minimi0ingthe area of the contour and e*entuall$ the cost function and can-e ignoredJsince it has no ph$sical meaning.


76/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSy

stemes

Static rior1 Concept $emonstration


77/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Static rior %continued&

%ne can also optimi0e the cost function 2ith respect to the unkno2nparameters of the morphing function

+eading to a nice 6selfsufficient7 s$stem of motion euations that

update the glo-al registration parameters -et2een the e*ol*ingcur*e and the model

,o2e*er the *aria-ilit$ of the model 2as not considered up to thispoint and areas 2ith high uncertainties 2ill ha*e the same impact onthe process


78/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Some 7esults %non2medical&


79/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Ta6ing Into Account the Model ncertainties

(a3imi0ing the &oint posterior @segmentation/morphingA is a uiteattracti*e criterion in 6inferencing7

Ghere the a$es rule 2as considered and gi*en that the

pro-a-ilit$ for a gi*en prior model is fi3ed and 2e can assumethat all @segmentation/morphingA solutions are euall$ pro-a-le2e get

)nder the assumption of independence...2ithin pi3elsJand thenfinding the optimal implicit function and its morphingtransformations is eui*alent 2ith


80/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Ta6ing Into Account the Model ncertainties

That can -e further de*eloped using the #aussian nature of themodel distri-ution at each image pi3el

" term that aims at reco*ering a transformation and a le*el setthat 2hen pro&ected to the model it is pro&ected to areas 2ithlo2 *ariance @high confidenceA

" term that aims at minimi0ing the actual distance -et2een thele*el set function and the model and is scaled according to themodel confidenceJ

2ould prefer ha*e a -etter match -et2een the model and le*el set in areas

2here the *aria-ilit$ is lo2

2hile in areas 2ith important de*iation of the training set this term 2ill -e

less important


81/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Ta6ing the derivatives'

Calculus of *ariations regarding the le*el set and the morphingfunction:

The le*el set deformation flo2 consists of t2o terms: that is a constant deflation force @2hen the le*el set function

collapses e*entuall$ the cost function reaches the lo2est potentialA

"n adapti*e -alloon @directional/magnitude2iseA force that

inflates/deflates the le*el set so it fits -etter 2ith the prior afterits pro&ection to the model spaceJIn areas 2ith high *ariance thisterm -ecome less significant and dataterms guide the le*el set to thereal o-&ect -oundaries...

n


82/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Comparative 7esults'

n


83/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Some 8ideos'%again non2medical&

n


84/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Some medical results

n


85/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Implicit Active Shapes.rousson2paragios:04

The "cti*e Shape (odel of Cootes et al. is uite popular to o-&ecte3traction. Such modeling consists of the follo2ing steps:

+et us consider a training set of registered surfaces@implicit representations can also -e used for registration F9HA.'istance maps are computed for each surface:

The samples are centered 2ith respect to the a*eragerepresentation :

n


86/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

Implicit Active Shapes.rousson2paragios:04

Training set:

The principal modes of variation are reco*ered throughPrincipal Component "nal$sis @PC"A. " ne2 shape can -egenerated from the @mA retained modes:

n


87/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

The model'

n


88/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformation

etSystemes

The prior

" le*el set function that has minimal distance from a linear fromthe model spaceJ

The unkno2n consist of: The form of the implicit function

The glo-al transformation -et2een the a*erage mode and the image

The set of linear coefficients that 2hen applied to the set of -asisfunctions pro*ides the optimal match of the current contour 2ith the

model space

"nd are reco*ered in a straightfor2ard manner using a gradientdescent methodJ


89/92

n


90/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformatio

etSystemes

Conclusions

PR%S

Elegant tool to track mo*ing interfaces

Implicit Cur*e Parameteri0ation O estimation of the geometricProperties

"-le to account 2ith topological changes a-le to descri-e multicomponent o-&ects

C%NS

Computational comple3it$

Numerical appro3imations redundanc$

%pen Cur*es sorr$ 2e C"NN%T do an$thing a-out thatJ

n


91/92

Centre

dEeignement

etde

Recherche

en

Technologies

de

LInformatio

etSystemes


"tlantis Research #roup"tlantis Research #roup

Ecole Nationale des Ponts et ChausseesEcole Nationale des Ponts et ChausseesParis !ranceParis !rance

Stanle$ %sherhttp://math.ucla.edu/~s&o

'epartment of (athematics'epartment of (athematics)ni*ersit$ of California +os "ngeles)ni*ersit$ of California +os "ngeles

)S")S"

,ttp://cermics.enpc.fr/~paragios/-ook/-ook.html

on


92/92

dEeignement

etde

Recherche

en

Technologies

de

LInformatio

es

7esources

ooks ames Sethian @>DD?>DDDA: +e*el Set O !ast (arching (ethodsCam-ridge Introductor$.

Stan %sher O Ronald !edki2 @4554A: +e*el Set (ethods and '$namicImplicit Surfaces Springer Introductor$.

Stan %sher O Nikos Paragios @4558A: #eometric +e*el Set in Imaging

Lision and #raphics Springer J(ostl$ LisionJ-it ad*ancedJ

People Fnone3clusi*e listH +aurent Cohen @medicalA 'a*id reen @graphicsA Rachid 'eriche

@segmentation tracking 'TIA Eric #rimson @medicalA %li*ier !augeras@stereoA Renaud 1eri*en @stereo segmentationA Ron 1immel@segmentation shape from shading trackingA err$ Prince @topolog$preser*ingA #uillermo Sapiro @segmentation tracking implicit surfacesAames Sethian a-a Lemuri @'iffusion Segmentation RegistrationAoachim Geickert @diffusion segmentationA Ross Ghitaker @#raphicsA

Levelset Tutorial

Documents

Transcript of Levelset Tutorial