Intensity-Based Registration CAMP II –...

Intensity-Based Registration

CAMP II – IN2022

Christian Wachinger, Nassir Navab

Computer Aided Medical Procedures (CAMP)

Technische Universität München, Germany

Intensity-Based Registration - C. Wachinger, N. Navab 2

Agenda

Introduction

Classification

Dimensionality / Modalities

Transformation

Registration Basis

Method

Similarity Measures

Interpolation, Multi-Resolution

Motivation

• Today, medical images are to a large extent digital

• Different images contain complementary information

• To use this information => Registration

Registration: Definition

Bring two images into spatial alignment, or establish a common geometric reference frame.

Intensity-Based Registration - C. Wachinger, N. Navab 5Source: Xenios Papademitris

Medical Imaging Modalities

MRICT PET 3D-US X-Ray US Video

Registration

Visualization of fused data

Measurement of changes Computer aided diagnosis / therapy

Intraoperative Navigation …

Agenda

Introduction

Classification

Transformation

Registration Basis

Method

Similarity Measures

Classification Scheme

Dimensionality:

Registration Basis:

Transformation:

Modalities:

Subject/Object:

3D-3D, 2D-3D, 3D-4D, …

Extrinsic (e.g. markers) vs. Intrinsic

Rigid, affine, projective, deformable

mono-modal / multi-modal

Intra- vs. Intersubject, which anatomy

Number of Images: Pair-wise, Group-wise

Dimensionality: 2D-2D / 3D-3D

• Digital Subtraction Angiography:X-Ray or CT scan image is subtracted from a second image with contrast agent >> highlighting of blood vessels

3D CT DSA:blood vessels

in the brain

2D X-Ray DSA

• Even if the same imaging device is acquiring the two images, registration is necessary in order to compensate for small shift of the patient position and movement of organs.

Dimensionality / Modalities: 3D-3D

PET-MRMR-CT

CT Scan

MR Scan

C-Arm Fluoroscopy

Registration →intraop. Navigation

CT Scan

Linear Accelerator

Registration →Patient Positioning

Dimensionality / Modality : 2D – 3D

CT Scan

Ultrasound

Registration →intraop. Navigation

US segmentation with registration to segmented CT

Ultrasound CT US-Segmentation

Registration Transformation

Image Y

Rigid Affine Projective Deformable

⎥⎦

⎤⎢⎣

⎡vTvtA

⎥⎦

⎤⎢⎣

⎡10tA

T⎥⎦

⎤⎢⎣

⎡10tR

T ?H4x4

Registration Basis - Extrinsic

Stereotactic Frame>> invasive method!

Markerse.g. invasive bone fiducials,noninvasive skin markers

Results in 3D point sets available for registration

Registration Basis - Intrinsic

anatomical landmarks>> 3D point sets to register

segmented objects>> 3D surfaces to register

full image content>> pixel/voxel intensities

Number of Images

• Pair-wise– Aligning two images

• Group-wise– Mosaicing

– Population studies, Atlas construction

Number of Images

• Pair-wise– Aligning two images

• Group-wise– Mosaicing

– Population studies, Atlas construction

Source: S. Joshi

Before Alignment

After Alignment

Source: L. Zöllei

Agenda

Introduction

Classification

Transformation

Registration Basis

Method

Similarity Measures

Intensity- vs. Feature-Based Registration

• Next class: Registration based on Features

• This class: Registration based on image intensities

Corresponding Points Point Cloud ↔ Shape

Intensity-Based 3D-3D Registration

X Y X,T(Y)• Define Transformation T on one of the images

• Compare X, T(Y) using the full image content• Refine T and repeat, until convergence criteria reached.

MRI fMRI

Similarity MeasurementTransformation T

Volume Y Volume X

Initial T

Interpolation

Similarity Measurement

Optimization

Transformation T

Volume Y Volume X

Iterative Execution

Initial T

Source: Xenios Papademitris

Interpolation

Optimization

Transformation T

Volume Y Volume X

Iterative Execution

Initial T

• In each iteration, compute a similarity measure sim(X, T(Y)) using the full image content of X and Y, i.e. all voxels!

• Using T(Y) requires voxel interpolation. Consideration of speed / quality!

• Maximize sim(X, T(Y)) by varying transformation→ optimization algorithm on transformation parameters!

• 3D Rigid transformation has 6 DOF

– 3 Translation Tt

– 3 Rotation Tr

Interpolation

Optimization

Transformation T

Volume Y Volume X

Iterative Execution

Initial T

How to represent the rotation?

T = Tt * Tr

T̂ = argmaxTsim(X,T(Y ))

Representations of 3D Rotations

• Rotation group SO(3) (special orthogonal group)

• 3 degrees of freedom

• Rotation matrix

• Axis angle

• Quaternion

• Euler angles

• …

SO(3) = { Q ∈ R3×3 : QQ> = I, det(Q) = 1}

Source: www.euclideanspace.com

⎡⎣cos γ − sin γ 0sin γ cos γ 00 0 1

⎤⎦⎡⎣1 0 00 cosβ − sinβ0 sinβ cosβ

⎤⎦⎡⎣cosα − sinα 0sinα cosα 00 0 1

⎤⎦

• In each iteration, compute a similarity measure sim(X,T(Y)) using the full image content of X and Y, i.e. all voxels!

• Using T(Y) requires voxel interpolation. Consideration of speed / quality!

• Minimize sim(X, T(Y)) by varying transformation→ optimization algorithm on transformation parameters!

Similarity Measures: SSD / SAD

Volume X Volume T(Y)

∑ −=i

ii yxN

SSD 2)(1

Sum of Squared Differences:Simple measure, can be optimized

with special algorithms

∑ −=i

ii yxN

Sum of Absolute Differences:Less sensitive on large intensity

differences than SSD

→ Only for intra-modal Registration, e.g. CT-CT

Similarity Measures: SSD / SAD

• Images slightly tilted:

• However: We encounter problems already if the images have different contrast or window/level values

Correlation Coefficient (Normalized Cross Correlation)

∑ −−=i

yyxxNCC ))((1σσ

Normalized Cross Correlation:Expresses the linear relationship between voxel

intensities in the two volumes

→ Already looser dependence on intensities!

Correlation Coefficient – Example

• Two random variables X and Y

• What is their Correlation Coefficient CC(X,Y) ?

• X uniformly distributed in [-1;1], Y = X2, CC(X,Y) ?

X 3 5 7 9 11Y 11 19 27 35 43

Elements

X -1 -0,9 -0,8 -0,7 -0,6 -0,5 -0,4 -0,3 -0,2 -0,1 0Y 1 0,81 0,64 0,49 0,36 0,25 0,16 0,09 0,04 0,01 0

Elements

0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 10,01 0,04 0,09 0,16 0,25 0,36 0,49 0,64 0,81 1

Correlation Coefficient - Example

• Four datasets, two variables X,Y, NCC(X,Y) = 0.81 for each.

Source: wikipedia

Correlation Coefficient - Example

• Point sets (x,y), with correlation coefficient of x and y.

Source: wikipedia

Multi-modal 3D-3D Registration

No simple relationship between X and Y intensities anymore:

Two approaches:

- Simulate one image from the other → reduce it to the mono-modal case

- Use more sophisticated similarity measures

• How to measure the relationship between X and Y ?

• Measure the structure of the joint distribution

• How to measure the structure?

• Shannon Entropy

• Optimum at a highly structured (clustered) histogram

Information Theoretic Approach

Shannon Entropy, developed in the 1940s(communication theory)

∑−=i

ii ppH log

piuniform distribution→ maximum entropy

any other distribution→ less entropy

Information Theoretic Approach - Entropy

• Images X and Y are treated as random variables

• px(i) = probability of a pixel of X having the intensity value i

• px(i) and py(i) can be estimated from histograms

i=0…255

Information Theoretic Approach - Entropy

Joint Entropy

• At each voxel location, we have a pair of intensity values, representing the combined image.

∑∑−=i j

xyxy jipjipYXH ),(log),(),(

Joint Entropy

pxy(i,j)

X and Y identical X and Y misaligned

Joint Histogram

Intensity of Reference X

Intensity of TransformedTarget Y

SSD OptimumY = X

NCC OptimumY = a*X + b

Joint Entropy

• Histogram for images from different Modalities

Joint HistogramTarget ImageSource Image

Not Aligned

Aligned

Source: PhD Thesis, L. Zöllei [7]

Mutual Information

• Maximized if X and Y are perfectly aligned

• Any permutation on the intensity values of X or Y does not affect the measure → great for multimodal registration!

• H(X) and H(Y) help to make the measure more robust

∑∑=

=−+=

i j yx

xyxy jpip

jipjip

YXHYHXHYXMI

)()(),(

log),(

),()()(),(

Agenda

Introduction

Classification

Transformation

Registration Basis

Method

Similarity Measures

Image Grid

Origin

Spacing in y

Spacing in x

An Image is sampling of a continuous field using a discrete grid

Source: ITK registration methods

Image Grid

Pixel Value

Pixel Region

[0,0] [1,0] [2,0]

Partial Volume Effect: a single voxel contains a mixture of multiple tissue values

Fixed Image Grid

xFixed Image

Physical Coordinates

x’Moving Image

Physical Coordinates

Moving Image Grid

Space Transform

Things I will not do …

I will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pix

256 x 256 pixels

MRI-T2

128 x 128 pixels

Scaling Transform

What scale factor ?a) 2.0b) 1.0c) 0.5

Things I will not do …

I will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pixel spaceI will not register images in pix

Interpolation Methods

• Essential Problem: Transforming an Image while keeping the rasterization grid

→ →

I1 T(I1) I2=T(I1)

Interpolation – Forward Warping

I2(u’,v’) I2(u’+1,v’)

I2(u’,v’+1) I2(u’+1,v’+1)

→ Holes possible

Interpolation – Backward Warping

⎟⎟⎠

⎞⎜⎜⎝

⎛′′

=⎟⎟⎠

⎞⎜⎜⎝

⎛ −

))(),((),( 12 yroundxroundIvuI =′′

I2(u’,v’)

I1(u,v)

I1(u+1,v)

I1(u+1,v+1)

I1(u,v+1)

(x,y) Nearest Neighbor Interpolation:

Bilinear Interpolation:

)1)(1)(1,1()1)()(1,())(1)(,1(

))()(,(),(

yvxuvuIyvuxvuIvyxuvuI

vyuxvuIvuI

−+−++++−+−++−−++

+−−=′′

Interpolation by Convolution

• Expression as convolution with a filter kernel h

Nearest:

Bilinear:

-2 -1 0 1 2-0.2

1 nearestlinearcubic

Bicubic:

Interpolation in 3D

• Extension to 3D: Trilinear Interpolation

Vxyz = V111 · (x− 1)(y − 1)(z − 1)+V110 · (x− 1)(y − 1)z+V101 · (x− 1)y(z − 1)+V011 · x(y − 1)(z − 1)+

V100 · (x− 1)yz+V010 · x(y − 1)z+V001 · xy(z − 1)+

V000 · xyz

Multi-Resolution (multi-scale) Approach

Registration

Fixed Image Moving Image

Student Project : Simultaneous Deformation

• Combination of registration of

– multiple images: simultaneously

– deformable registration

The end …

Thank you for your attention!

Intensity-Based Registration CAMP II –...

Documents

Transcript of Intensity-Based Registration CAMP II –...

Traffic Intensity

BrainPrint: A Discriminative Characterization of Brain ...reuter.mit.edu/blue/papers/wachinger-brainprint15/wachinger-brainprint15.pdf · identi cation, (ii) age and sex prediction,

Intensity, Intensity Level, and Intensity Spectrum Level The spectrum level is the intensity level of the sound wave within a 1 Hz band.

InTENSity EX4 - LSI International International - InTENSity EX4.pdf · The InTENSity™ EX4 is an ... 1s, 2s, 5s Cycle ... DQ7000 InTENSity™ EX4 Clinical Multifunctional Electrotherapy

Poisson Intensity Estimation with Reproducing Kernels · Poisson Intensity Estimation with ... spatial statistics, intensity estimation, ... Flaxman et al./Poisson Intensity Estimation

JASS `04 Benjamin Fingerle, Christian Wachinger1 2nd Joined Advanced Student School Ubiquitous Tracking based AR-Setups Benjamin Fingerle Christian Wachinger.

Infinite Intensity

Intensity Manual

Plyometric Workout Low intensity and High intensity workouts.

The 2D Analytic Signal for Envelope Detection and …people.csail.mit.edu/wachinger/articles/MedIA-Analytic-2012.pdf · The 2D Analytic Signal for Envelope Detection and Feature Extraction

Presented By Kanakamadala Bharath Kumar Mat Nr: 231951heindl/ebte-09ss/... · Kanakamadala Bharath K 30‐July‐2009 umar 2. 1.0 YouTube History 4 ... Kanakamadala Bharath Kumar

Light Intensity D. Crowley, 2008. Light Intensity To understand how light intensity affects photosynthesis Friday, September 11, 2015.

Sound Intensity

BrainPrint : Identifying Subjects by their Brainreuter.mit.edu/blue/papers/brainprint-miccai14/brainprint-miccai14.pdfBrainPrint: Identifying Subjects by their Brain Christian Wachinger

lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity

EPIV 09SS SolSt K3 Lattice Vibrations

Early Childhood Service Intensity Instrument · 2015-07-08 · Service Intensity Level I: Minimal Service Intensity Beginning Care. 1) Level of intensity of services needed. 2) Types

Variation of Light Intensity: Measuring the Light Intensity of Different ...

Intensity, Intensity Level , and Intensity Spectrum Level

UNCERTAINTY ESTIMATION IN INTENSITY …lcr.uerj.br/Manual_ABFM/Uncertainty Estimation in Intensity... · UNCERTAINTY ESTIMATION IN INTENSITY-MODULATED RADIOTHERAPY ABSOLUTE DOSIMETRY