PhD Thesis Michael Baumann Supervisors Jocelyne Troccaz Vincent Daanen

A 3D ULTRASOUND-BASED TRACKING SYSTEM FOR PROSTATE BIOPSY DISTRIBUTION QUALITY

INSURANCE AND GUIDANCE.

PhD Thesis

Michael Baumann

Supervisors

Jocelyne TroccazVincent Daanen

22

Context of this thesis

TIMC laboratory• specializing in computer-assisted medical interventions for more than

twenty years now• many clinical and industrial collaborations

Pitié-Salpétrière hospital, urology department• active support of this work and very inspiring exchanges• clinical data acquisition on more than 70 patients now

Koelis SA• industrial partner• objective: commercialize products based on prostate tracking

Registration FrameworkRegistration Framework

○○○○○○○○○○○○○○Experiments and Experiments and ResultsResults

○○○○○○

DiscussionDiscussion

○○

OutlineOutline

Conclusion/ApplicationsConclusion/Applications

○○○○

¯̄IntroductionIntroduction

●○○○●○○○Prostate Tissue Prostate Tissue TrackingTracking

○○○○

33

Introduction

Prostate

Prostate Cancer• most frequent cancer in men

- ~220.000 new cases in US (2007)- ~345.000 new cases in EU25

(2006)• second cause of cancer death for men

- 27.000 deaths in US (2007)- 87.400 deaths in EU25 (2006)

• slow growing disease• affects mostly elder men (>50 years)

Bladder

Seminal Vesicles

Rectum

Prostate

Urethra



○○○○○○


○○

OutlineOutline


○○○○



○○○○

44DRE

Introduction

Prostate Specific Antigene (PSA) screening• biological tumor marker• sensitivity for 4ng/ml threshold: 68-83% (clinically significant

cancer)• specificity: ~30% false positives!

Digital Rectal Exams (DRE)• highly varying sensitivity in clinical studies reported: 18% to

68%• specificity: 4% to 33%• complementary to PSA screening

Prostate Biopsies• Sensitivity: 60-80 % (clinically significant cancer)• Specificity: >95% (histological analysis)• invasive programmed only if DRE/PSA positive• dilemma: false negatives repeated biopsies



○○○○○○


○○

OutlineOutline


○○○○



○○○○

55

Prostate Biopsies• 2D transrectal ultrasound (TRUS) control• needle guide on probe

• guide aligned with longitudinal plane of probe

2D TRUS probe with needle guide

Introduction

longitudinal cut

corresponding 2D US image

with needle trajectory



○○○○○○


○○

OutlineOutline


○○○○



○○○○

66

Introduction

Biopsy targets• prostate cancer is isoechogenic

systematic targets• McNeal’s 3-zone model: central zone (CZ),

transition zone (TZ), peripheral zone (PZ)• 68% of cancer can be found in peripheral

zone

Systematic 12-core protocol

• clinical representation in (pseudo-)coronal plane

coronal plane

coronal plane



○○○○○○


○○

OutlineOutline


○○○○



○○○○

77

Prostate Motion Problem

Prostate motion• main challenge for any prostate tissue tracking system• displacements and deformations

Transrectal biopsy specific: probe-related motion• end-fire probe• deformations and displacements due to probe pressure

Neighboring organs (diaphragm motion, rectal and bladder filling)• minor impact during prostate biopsies



○○○○○○


○○

OutlineOutline


○○○○


●●○○●●○○Prostate Tissue Prostate Tissue TrackingTracking

○○○○

88

Prostate Motion Problem

Patient motion• (small) deformations

• displacements with respect to surrounding tissues

• displacements with respect to operating room (pelvis movements!)



○○○○○○


○○

OutlineOutline


○○○○


●●○○●●○○Prostate Tissue Prostate Tissue TrackingTracking

○○○○

99

Introduction

Biopsy and Target Localization Problem• only rudimentary knowledge about biopsy

position• at all stages of intervention!

Pre-interventional stage/planning• n-core protocol target definition highly

approximate• targets have to be mentally mapped into patient

anatomy

During intervention: target localization problem• difficult to aim invisible target under 2D control

- ultrasound: few structural information- 2D: no depth information- prostate motion

finding the target : what do we aim exactly?



○○○○○○


○○

OutlineOutline


○○○○


●●●○●●●○Prostate Tissue Prostate Tissue TrackingTracking

○○○○

1010

Introduction

Target localization problem (ctd)• there exist better targets than systematic protocol• high quality cancer distribution atlas available [Shen’01]

- simulations: biopsy sensitivity > 96% with only 6 needles (transperineal access)

• suspicious lesions identified on IRM• repeated biopsy series

- avoid already sampled tissues (negative targets)

• how to aim these targets?

After intervention : sample localization problem• where were the samples taken exactly?• quality control?

- are there unsampled regions?• difficult to map histological cancer information back to anatomy

difficult to use histological information for focal treatment planning



○○○○○○


○○

OutlineOutline


○○○○


●●●○●●●○Prostate Tissue Prostate Tissue TrackingTracking

○○○○

1111

Magnet Resonance imaging-based approaches• objective : target suspicious lesions detected on MR images• biopsy under MRI control• instruments calibrated with MR frame

Beyersdorff [05], Musil, Krieger et al. [04,05,07], Stoianovici [07]• IRM compatible biopsy acquisition instruments/robot

• pro: possibility to aim IRM targets• con: cannot detect/compensate patient movements

- would require high resolution, real-time MRI• con: diagnosis: cost-benefit ratio unsatisfying

- several millions of biopsies/year in US and EU

Existing Solutions



○○○○○○


○○

OutlineOutline


○○○○


●●●●●●●●Prostate Tissue Prostate Tissue TrackingTracking

○○○○

1212

Probe tracking + registration based approaches• 2D transrectal ultrasound• track US probe with optical or magnetic tracking system

- identifies view cone motion• register 2D tracking images with free-hand reference volume

- identifies prostate motion

Xu et al. [07]: Magnetic probe tracking + registration• pro: can compensate smaller rigid prostate-movements• con: free-hand volume with end-fire probe low accuracy• con: rigid registration• con: registration of lateral biopsy images not robust (partial

gland problem)• con: difficult to compensate large pelvis movements

Existing Solutions



○○○○○○


○○

OutlineOutline


○○○○



○○○○

1313

Outline of the Presentation

Prostate Tissue Tracking and Guidance• clinical and scientific objectives• soft-tissue tracking

Prostate Image Registration• registration framework• multi-resolution techniques• image distance metric (rigid)• probe movement model• rigid refinement• elastic registration framework• forces for elastic registration

Experiments and Results• registration success rate• accuracy• biopsy maps and targeting accuracy study

Discussion Conclusion and Potential Applications

• clinical and scientific contributions• potential applications



○○○○○○


○○

OutlineOutline


○○○○



○○○○

1414

Objectives

Scientific objectives• prostate tissue tracking

- establish tissue correspondence- with respect to a reference space

• goal: establish correspondence between- biopsy site planning- reference space- needle position during intervention

Clinical objectives• more sophisticated targets

- MRI, statistical cancer atlas, unsampled zones when repeating biopsies

• guide clinician to target• feed-back to clinician about exact sample position

- immediately and after intervention• biopsy maps

¯̄Prostate Tissue TrackingProstate Tissue Tracking

●○●○

IntroductionIntroduction

●●●●●●



○○○○○○


○○

OutlineOutline


○○○○

1515

guidance:

target projection into tracking volume

anchor volume:

acquired before intervention

defines the reference space

tracking volume:

acquired during intervention:

“contains” sample trajectory

needle projection:

projection into anchor volume

projection can lead to curbed trajectories

registration:

establishment of correspondences for identical tissues present in both images

Image-based Prostate Tracking Framework

Proposed Solution• 3D ultrasound-based• hybrid registration

- image-based- a priori model based

• deformation estimation• no probe tracking• miniminal overhead for clinician, no segmentation

3D ultrasoundview cone

anchor volume tracking volume

biopsy map:

contains projections of all samples

¯̄Prostate Tissue TrackingProstate Tissue Tracking

●●●●


●●●●●●



○○○○○○


○○

OutlineOutline


○○○○

1616

Registration Framework

Image Registration• Optimization (minimization) problem

• φ = transformation model• T = template/transformed image (R3 R)• R = reference/fixed image

• D[.] = cost functional

Problems• registration only efficient with local minimization (downhill

search)• successful local minimization requires

- locally unimodal cost functional- start point inside the convex region

• the more degrees of freedom (DOF) of φ, the more difficult to find unimodal region of D[R,T,φ] !

OK

KO KO


●●●●●●Prostate Tissue Prostate Tissue TrackingTracking

●●●●

Experiments and Experiments and ResultsResults

○○○○○○


○○

OutlineOutline


○○○○

¯̄Registration FrameworkRegistration Framework

●○○○○○○●○○○○○○

1717

3 DOF 6 DOF ~ 125.000 DOF

voxel intensity based image distance metrics

a priori models

multi-resolution approaches

optimization techniques

Proposed approach


Probe kinematics based rigid presearch

Refinement of rigid estimate Elastic

estimation

multivariate correlation coefficient

parametric systematic

search

SSD with local intensity shift

loss-containing multi-resolution techniques

endorectal probe

kinematics

bio-mechanical probe insertion

parametric local

optimization

variational optimization

inverse consistency

linear elasticity



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●○○○○○○●○○○○○○

1818

Multi-Resolution

Multi-resolution approach• Gaussian pyramid• registration performed on different resolution levels

Coarse resolutions and information loss

probe kinematics

rigid refinement

elastic

?

70% 56%

≤ 50% of fine-grid voxel mask coarse-grid voxelelse use average of available voxel

attention: introduces, however, local information shifts

US-specific: complex image masks

problematic when computing level n+1 from level n

level n level n+1



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●○○○○○●●○○○○○

70% 71%

1919

Multi-resolution

50-percent rule• use it for pyramid construction

• for interpolation

• for every other computation on multiple voxels- gradient computation (image distance metrics!)- Gaussian smoothing

Conclusion• Makes high-speed volume to volume registration possible

- reliable registration on very coarse levels• Disadvantage

- introduces small local information shifts

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●○○○○○●●○○○○○

50% rule

level 5

standard

level 5

standard

level 5

50% rule

level 5

level 1

level 1

2020

Distance metric (rigid)

Image distance metric (Rigid Registration)• correlation coefficient (CC) based

- well-proven for monomodal registration• multivariate application

- intensity image + gradient magnitude image- more robust results on coarse levels

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●○○○○●●●○○○○

raw image gradient magnitude

2121

Challenge• probe used to guide needle view cone motion• adds up to prostate motion

• motion too large for capture range of image distance metric- direct downhill/local registration only ~30-40% success rate

Observations• probe head always in contact with rectal wall in front of prostate

- if not, no prostate image or needle trajectory outside prostate

• anal sphincter heavily constrains probe motion- fix point for probe motion

• most important rotations occur around probe axis (when switching lobe)

Probe kinematics

0°-30° 180°

0° 180 360°

OK

KO KO

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●○○○●●●●○○○

2222

Probe kinematics

Model of endorectal probe kinematics:

• approximate prostate capsule with ellipsoid from bounding box• estimate rectal probe fix point• admit only positions for which

- the probe axis lies on the fix point- the probe origin lies on the membrane

• 3 degrees of freedom only- can be exhaustively explored in reasonable time!

Advantages• Makes solution independent of external tracking system!• Solves patient motion problem!

PrSurf(0,0)

CPro

Bounding Box

Probe position in reference image

FPRect

Bounding Box

Prostate Ellipsoid

β PrSurf(α,β)

OUS

Bounding Box

Prostate Ellipsoid

λ

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●○○○●●●●○○○

2323

Rigid Refinement

Refinement of rigid estimate

rigid registration of 5 best transformations

provided by probe kinematics

high quality registration of best result

probe kinematics

rigid refinement

elastic

high quality local search:

from coarse to fine

high speed:

optimize on coarsest level

classical local/downhill search algorithm:

Powell-Brent



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●○○●●●●●○○

2424

Elastic Registration

Prostate deformations• relatively small (several millimeters)• strongest near probe head• difficult to estimate:

- few image information near probe head

Transformation model: displacement field

Framework

• : linear elastic potential– regularizes/smoothes displacement field– minimal when no deformation strong regularizer

• : SSD variant to measure image distance• : bio-mechanical simulation of probe insertion• : inverse consistency constraints

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●○●●●●●●○

2525


Elastic regularization: solution scheme• variational approach• necessary condition for solver u* of cost function:

- Gâteaux derivative at u* vanishes for all perturbations ψ• Euler-Lagrange equations for linear elastic regularization:

• trick: separate force computation and regularization1. accumulate forces2. solve Euler-Lagrange equations

• then we get an elliptic boundary value problem of the form

• trick: introduce artificial time to obtain iterative gradient descent scheme

probe kinematics

rigid refinement

elastic

gradient of linear elastic potential

gradients of distance metrics



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●○●●●●●●○

2626

Force terms

Image based forces• correlation coefficient: statistically not robust when locally

computed• SSD

• assumes identity between R and Tû

- does not correspond to reality!- changes in ultrasound gain, probe pressure and ultrasound

direction

Local intensity shift model• additive model:• b estimated with Gaussian convolutions on R and T

• resulting force term

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●●●●●●●●●

2727

Force terms

Bio-mechanical probe insertion model• model of probe-related tissue displacements

• Interpret displacement differences as forces in the estimation process

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●●●●●●●●●

2828

Force terms

Inverse consistency forces• Observation: forward and backward estimation u and v not

symmetric:

• Zhang’s approach [’06]- estimate u and v simultaneously- enforce inverse consistency by minimizing- alternating optimization process:

• resulting force term

u

v

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●●●●●●●●●

2929

Proposed approach


Probe kinematics based rigid presearch

Refinement of rigid estimate Elastic

estimation

multivariate correlation coefficient

parametric systematic

search

SSD with local intensity shift

loss-containing multi-resolution techniques

endorectal probe

kinematics

bio-mechanical probe insertion

parametric local

optimization

variational optimization

inverse consistency

linear elasticity



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●●●●●●●●●

3030

Experiments and results

3131

Experiments and Results

Experiments• on real patient data• Pitie-Salpétrière Hospital, Paris, urology department

- P. Mozer, G. Chevreau, S. Bart, J.-C. Bousquet• 3D ultrasound images (GE Voluson, RIC5-9 probe)

- acquired before biopsies and after each sample acquisition- targeting carried out under 2D US control

Registration example


●●●●●●Prostate Biopsy Prostate Biopsy TrackingTracking

●●●●


●●●●●●●●●●●●●●


○○

OutlineOutline


○○○○

¯̄Experiments and ResultsExperiments and Results

●○○●○○

3232


Rigid Registration • Algorithm tested on 785 image pairs from 47 patients• 27 mis-registrations (success-rate 96.5 %)

Conclusion• probe movement model works fine!

ultrasound depth

ultrasound quality

partial contact

PrSurf(0,0)

CPro

Bounding Box

Probe position in reference image

FPRect

Bounding Box

Prostate Ellipsoid

β PrSurf(α,β)

OUS

Bounding Box

Prostate Ellipsoid

λ



●●●●


●●●●●●●●●●●●●●


○○

OutlineOutline


○○○○


●○○●○○

3333


Accuracy study• 208 registrations on data from 14 patients• manual point fiducial segmentation (calcifications, dark spots)• error computed on Euclidean distances of corresponding

fiducials

Registration accuracy

• rigid optimization performed on resolution levels 5 to 3• elastic optimization performed on resolution levels 6 to 3

Conclusion• accuracy sufficient for many clinical applications

rigid elasticfiducial distances (RMS) 1,41 mm 1,10 mmfiducial distances (max) 3,84 mm 2,93 mmexecution time 6,5 s 16,7 s



●●●●


●●●●●●●●●●●●●●


○○

OutlineOutline


○○○○


●●○●●○

3434


First Application: Biopsy maps• show targeting difficulties

P. Mozer, M. Baumann, G. Chevreau, A. Moreau-Gaudry [Mozer’08]• apex and base targets more difficult to reach

than central gland• operator learning curve measured



●●●●


●●●●●●●●●●●●●●


○○

OutlineOutline


○○○○


●●●●●●

3535

Discussion

3636

Discussion

Automatic registration validation• visual validation time-consuming and operator-dependent• open issue: automatic detection of failures!• necessary for guidance!

Registration and real-time• requires 5 – 15 seconds• stream parallelization:

- algorithm mainly consists of image convolutions- can be parallelized on a voxel per voxel basis- well suited for latest graphic card architectures (stream

processors)• registration times of 1 second or less should be feasible

Similarity measures• Good performance for intra-series registration• Still to be evaluated for inter-series registration

- only one patient with two biopsy series for instance• Intensity shift model

- depends strongly on parameter σ of Gaussian convolution



●●●●


●●●●●●●●●●●●●●

OutlineOutline


○○○○

¯̄DiscussionDiscussion

●●


●●●●●●

3737

Discussion

Probe movement model (rigid registration)• very good success rate

- no probe tracking necessary less hardware in OR! Simpler workflow and logistics!

• improvements with model to data fitting possible• should further improve success rate

Bio-mechanical probe insertion model (elastic registration)• for about 50% image pairs, the model improves elastic

registration• but: sometimes inadequate model of reality



●●●●


●●●●●●●●●●●●●●

OutlineOutline


○○○○


●●


●●●●●●

3838

Discussion

Clinical acceptability• only slight modification of classical acquisition protocol

- bounding box placement- registration validation (probably post-op step)

• no additional instruments/hardware in operation room• cost effective: cost similar to current procedure



●●●●


●●●●●●●●●●●●●●

OutlineOutline


○○○○


●●


●●●●●●

3939

Conclusion and Applications

4040

¯̄Conclusion/ApplicationConclusion/Application

●○●○

Conclusion

Scientific contributions• probe movement model

- robust- no probe tracking hardware required- completely solves patient movement problem- reusable for many endocavitary US interventions!

• loss-containing multi-resolution filtering and interpolation- robust optimization on very sparse resolution levels

• hybrid model- and image-based elastic deformation framework• novel voxel similarity measure for elastic registration

- remarkably robust- simple

• proof of concept on large set of patient data

Medical contributions• biopsy accuracy study on biopsy maps

- more difficult to reach apex/base than mid-gland• operator learning curve proven



●●●●


●●●●●●●●●●●●●●

OutlineOutline


●●


●●●●●●

4141

Future work/Prospects

Potential Applications : Biopsies• biopsy maps

- immediate feed-back, post-interventional quality control• cancer maps

- map histological results on 3D biopsy map• guidance

- assist clinician during targeting- requires automatic registration validation and real-time

registration• guidance MRI target mapping

- reach MRI targets under ultrasound control- requires

– MRI to ultrasound registration• guidance repeated biopsy series

- avoid multiple sampling- visualize already sampled tissues

• guidance cancer atlas targets- define targets with cancer probability atlas (Shen’01)- map them onto anchor volume

– requires atlas to ultrasound volume registration



●●●●


●●●●●●●●●●●●●●

OutlineOutline


●●


●●●●●●

¯̄Conclusion/ApplicationsConclusion/Applications

●●●●

4242

Future work/Prospects

Potential applications : Therapy• improve accuracy of ultrasound-guided therapy

- brachytherapy, HIFU, cryotherapy, …• focal therapy?

- currently: two unknowns after positive biopsy findings1. shape of the tumor2. exact location of the biopsy

- not accurate enough for focal therapy- we solve 2!- sufficient for focal therapy?

– in combination with statistical tumor atlas?



●●●●


●●●●●●●●●●●●●●

OutlineOutline


●●


●●●●●●

¯̄Conclusion/ApplicationsConclusion/Applications

●●●●

4343

Publications and References

Publications [Baumann’07] M. Baumann, P. Mozer, V. Daanen, J. Troccaz. Towards 3D

Ultrasound Image Based Soft Tissue Tracking: a Transrectal Ultrasound Prostate Image Alignment System. MICCAI'07, Brisbane, Australia, 2007. Springer LNCS 4792.

[Mozer’07] P. Mozer, M.Baumann, G. Chevreau, J. Troccaz. “Fusion d’images : application au contrôle de la distribution des biopsies prostatiques,” Progrès en Urologie (les Cahiers de la Formation Continue), vol. 18 (1), 2008

[Baumann’08] M. Baumann, P. Mozer, V. Daanen, J. Troccaz. “Fast and robust elastic registration of endorectal 3D ultrasound prostate volumes for transrectal prostate needle puncture tracking,” In proceedings of CARS’08, Barcelona, 2008

References [Shen’04] D. Shen, Z. Lao, J. Zeng, W. Zhang, I. A. Sesterhenn, L. Sun, J. W.

Moul, E. H. Herskovits, G. Fichtinger, and C. Davatzikos. “Optimization of biopsy strategy by a statistical atlas of prostate cancer distribution,” Medical Image Analysis, vol. 8, no. 2, pp. 139–150, 2004.

[Zhang’05] Z. Zhang, Y. Jiang, and H. Tsui. “Consistent multi-modal non-rigid registration based on a variational approach,” Pattern Recognition Letters, pp. 715–725, 2006.

4444

Acknowledgements

Urology department Pitié-Salpétrière• Pierre Mozer, Grégoire Chevreau, Stéphane Bart

Koelis SA• Antoine Leroy • Vincent Daanen

TIMC• GMCAO group• Jocelyne Troccaz

and everyone else who supported this project during the last three years!

Funding: • 2004-06: ”Programme Hospitalier de Recherche Clinique -

Prostate-Echo”, French ministry of research• 2005-07: “Surgétique Minimalement Invasive (SMI)”, Agence

Nationale de Recherche (ANR)• 2005-08: Association Nationale de la Recherche Technique,

bourse CIFRE

4545

Inadequate probe model• possible explanation

ultrasound gel no elastic

deformation!!

gland not deformed at

all

4646

Separate elastic estimation• first step:

- estimate deformations caused by probe forces• second step:

- estimate deformations caused by image forces- start optimization with probe deformation as initial guess

4747

Elastic Regularization

Elastic regularization: solution scheme (ctd)• von Neumann stability analysis of numerical scheme yields

Stability criterion and elasticity parameters• forces in our framework are not physical

- derived from distances- how to calibrate them with the elastic forces?

• Young’s modulus E has no physical meaning- interpret it as free parameter- control elasticity parameters with Poisson’s coefficient v

and ∆t

• seek best balance between smoothness and convergence rate balance elastic smoothing and maximally admitted deformation

probe kinematics

rigid refinement

elastic

Young’s modulus

Poisson’s coefficient



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●○●●●●●●○

4848


Elastic regularization: solution scheme (ctd)• solved with Gauss-Seidel and full multigrid strategy

Boundary conditions• bending side-walls, fixed edges• good model for probe insertion

probe kinematics

rigid refinement

elastic



●●●●


○○○○○○


○○

OutlineOutline


○○○○


●●●●●●○●●●●●●○

4949

Biopsy acquisition• patient in dorsal or lateral

position• local anesthesia• 12 acquisitions

Introduction

PhD Thesis Michael Baumann Supervisors Jocelyne Troccaz Vincent Daanen

Documents

Transcript of PhD Thesis Michael Baumann Supervisors Jocelyne Troccaz Vincent Daanen