Registration by continuous optimisation

Stefan Klein

Erasmus MC, the Netherlands

Biomedical Imaging Group Rotterdam (BIGR)

Registration = optimisation

1

C

txty

Registration = optimisation

1

C

txty

Registration = optimisation

1

C

txty

Registration = optimisation

1

C

txty

Example

2

Example

2fixed image moving image

Example

2fixed image moving image

Example

2fixed image moving image

3

• F(x) = fixed image, M(x) = moving image

x = voxel coordinate

• Transformation function: T(x ; p)

p = vector of transformation parameters

• Cost function: C( p )

measures similarity of fixed image F(x) and

deformed moving image M( T(x; p) )

• Find p that minimises C

Math

4

Iterative optimisation

pk+1 = pk + ak . dk

dk = search direction

ak = step size

gradient descent:kkk )(

Cgp

pd

pk+1 = pk - ak . gk

P1 p1 g1

p2 p2 g2

p3 = p3 - ak . g3

::

::

::

k+1 k k

Gradient descent

5

pk+1 = pk - ak . gk

p1 p1 g1

p2 p2 g2

p3 = p3 - ak . g3

::

::

::

k+1 k k

=Cp1 k

Gradient descent

5

6

Cost function derivative

Example for mean of squared differences:

x

x

x

xp

TpxTx

ppxTx

p

pxTxp

M));((M)(F

N

2

M));((M)(F

N

2C

));((M)(FN

1)(C

t

2

7

Choice of dk

pk+1 = pk + ak . dk

Choice of dk

8

C

p1p2

gradient descent

Choice of dk

8

C

p1p2

smarter steps

Choice of dk

8

C

p1p2

cheaper steps

pk+1 = pk + ak . dk

gradient descent: dk = - gk

Newton: dk = - [Hk]-1 gk

quasi-Newton: dk = - Bk gk

conjugate gradient: dk = - gk + βk dk-1

stochastic gradient: dk - gk

Choice of dk

smarter steps

cheaper steps

9

10

Experimental comparison

Cardiac CT, 97x97x97 voxels, artifically deformed

11

Experimental comparison

Cardiac CT, 97x97x97 voxels, artifically deformed

12

Experimental comparison

Error measure:

x

xTxT )(ˆ)(N

1e

Experimental comparison

computation time

e [mm]

gradient descent

quasi-Newton

conjugate gradient

stochastic gradient

0

0.5

1

1.5

2

2.5

3

0.001 0.01 0.1 1 10 100 1000

14

Choice of ak

pk+1 = pk + ak . dk

Choice of ak

C

p1p2

15

Too small steps

Choice of ak

C

p1p2

15

Too large steps

16

Choice of ak

constant: ak = a

slowly decaying: ak = f ( k ) = a / ( A + k )a

exact line search: ak = argmina C ( pk + a dk )

inexact line search: ak argmina C ( pk + a dk ) [Wolfe conditions]

adaptive: ak = F ( progress in previous iterations )

pk+1 = pk + ak . dk

Stochastic gradient descent with

adaptive strategy for ak

)(sigmoidtt

)tA/(a)t(f

)t(f

1kTkk1k

kk

kkk1k

a

gg

gpp

-5 0 5

-1

0

1

17

0 250 5000

10

20

Stochastic gradient descent with

adaptive strategy for ak

)(sigmoidtt

)tA/(a)t(f

)t(f

1kTkk1k

kk

kkk1k

a

gg

gpp

-5 0 5

-1

0

1

17

0 250 5000

10

20

Choose a such that:

max. voxel displacement per iteration < [mm]

(with 95% probability)

Experimental comparison

• 6 prostate MR image pairs: nonrigid registration

• evaluation measure:

overlap of manual segmentations after registration

18

Experimental comparison

[mm]

A

non-adaptive2000

0.0

31

25

0.0

62

5

0.1

25

0.2

5

0.5

1.0

2.0

4.0

8.0

1.25

2.5

5

10

20

40

80

160

320

0.8

0.85

0.9

0.95

[mm]

A

adaptive2000

0.0

31

25

0.0

62

5

0.1

25

0.2

5

0.5

1.0

2.0

4.0

8.0

1.25

2.5

5

10

20

40

80

160

320

0.8

0.85

0.9

0.95

non-adaptive adaptive

19

Experimental comparison

20

Experiments with:

• brain, lung, prostate

• CT, MRI

• sum of squared differences, mutual information,

normalized mutual information

• rigid, nonrigid

size voxel ,20 A

good results in all experiments!

Local similarity measures

21

• MI = mutual information

• LMI = localised mutual information

=

assumes grey-value distribution does

not vary over image domain

)( x

xMIN

1 (aka: regional MI, conditional MI,

spatial information encoded MI)

Local similarity measures

21

• MI = mutual information

• LMI = localised mutual information

=

assumes grey-value distribution does

not vary over image domain

)( x

xMIN

1 (aka: regional MI, conditional MI,

spatial information encoded MI)

can be efficiently implemented with

stochastic gradient descent!

Summary

• Parametric formulation can be solved by continuous optimisation

• Derivative-based methods: require

• Extensive literature

• Basic method: gradient descent

• Popular choice:

quasi-Newton or conjugate gradient icm inexact line search

• “Recommended”:

stochastic gradient descent with adaptive step sizes

pC

22

Literature

• Nocedal & Wright: Numerical Optimization

• IEEE Trans. Image Processing 2007 - Klein, Staring, Pluim

Evaluation of optimization methods for nonrigid medical image registration using mutual

information and B-splines

• Int. J. Computer Vision 2009 - Klein, Pluim, Staring, Viergever

Adaptive stochastic gradient descent optimisation for image registration

• IEEE Trans. Image Processing 2000 - Thevenaz, Unser

Optimization of mutual information for multiresolution image registration

23

• Rigid and nonrigid registration

• Various cost functions, transformation models, multiresolution

strategies etc.

• Many optimisation algorithms implemented

• Free: http://elastix.isi.uu.nl

• Based on Insight ToolKit (ITK): http://www.itk.org

• IEEE Trans. Medical Imaging 2010 - Klein, Staring, Murphy,

Viergever, Pluim – elastix: a toolbox for intensity based medical image registration24