Clojure in the Cloud Everett Toews Developer JavaOne Sept. 29, 12:30 pm.
F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei,...
-
Upload
arleen-todd -
Category
Documents
-
view
212 -
download
0
Transcript of F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei,...
![Page 1: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/1.jpg)
FEATURE-BASED ALIGNMENTOF VOLUMETRIC MULTI-MODAL IMAGES
Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI
![Page 2: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/2.jpg)
Challenges: Arbitrary subjects, anatomies (brain, body, …), modalities (MR, CT, …), pathology, lack of one-to-one homology, unknown initialization, DICOM errors, …
ROBUST IMAGE ALIGNMENT
Thoracic CTInfant Brain MR
TBI, MR
Brain MR, CT(tumor)
2
![Page 3: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/3.jpg)
Robust clinical usage Initialize registration, segmentation routines
Large-scale data mining, e.g. Google style
APPLICATIONS
Thoracic CTInfant Brain MR
TBI, MR
Brain MR, CT(tumor)
3
![Page 4: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/4.jpg)
FEATURE-BASED ALIGNMENT METHOD
Alignment via 3D scale-invariant feature correspondences
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
4
![Page 5: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/5.jpg)
FEATURE-BASED ALIGNMENT METHOD
Strengths Robust: lack of one-to-one homology, disease, resection,
Globally optimal: no ‘capture radius’, initialization
Efficient: Memory, computation
Useful: Alignment, disease classification, prediction
Weaknesses Does not align different modalities Requires pre-aligned training data
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
5
![Page 6: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/6.jpg)
CONTRIBUTIONS
Inverted local feature correspondence Extend scale-invariant feature representation to
multi-modal alignment
Group-wise feature-based alignment Remove requirement for pre-aligned training
data Multiple modalities
6
![Page 7: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/7.jpg)
OVERVIEW
Scale-invariant feature representation
Inverted feature correspondence
Group-wise feature-based alignment
![Page 8: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/8.jpg)
OVERVIEW
Scale-invariant feature representation
Inverted feature correspondence
Group-wise feature-based alignment
![Page 9: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/9.jpg)
SCALE-INVARIANT FEATURES
Distinctive image patches Image-to-image
correspondence SIFT method: computer vision
Invariant to scaling, rotation, translation, illumination changes
Fast, efficient, robust Large-scale image search
Distinctive Image Features from Scale-Invariant KeypointsD. G. Lowe, IJCV, 2004.
9
![Page 10: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/10.jpg)
3D Geometry S = {X, σ, Θ} Location X = (x, y, z) Scale σ Orientation
(axis: 3 unit vectors )
Appearance I Intensity descriptor
SCALE-INVARIANT FEATURES IN 3D
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
S I
)ˆ,ˆ,ˆ( 321
10
![Page 11: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/11.jpg)
Difference-of-Gaussian scale-space extrema.
IDENTIFYING LOCATION X, SCALE σ
d
xdI ),()(xI
x
),()(),( xGxIxI
Feature locations, scales:blob-like image patterns
d
xdIx
),(localmax
,
iiX ,
11
![Page 12: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/12.jpg)
Dominant local image gradient directions)( IH 3D gradient orientation histogram
I
I
Vote location (upon unit sphere)
IVote magnitude
ASSIGNING ORIENTATION Θ
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
)(argmaxˆˆ
1 IH
))ˆ(ˆ( argmaxˆ11
ˆ2
IH 213
ˆˆˆ
)(XII )( IH
12
![Page 13: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/13.jpg)
Encode local image content For image correspondence / matching
Gradient orientation histogram (GoH) Quantization: 8 location x 8 orientation
bins 113 voxels → 64 bins (small size)
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
FEATURE DESCRIPTOR I
IGoHNormalized
Image PatchI
)(XI
13
![Page 14: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/14.jpg)
OVERVIEW
Scale-invariant feature representation
Inverted feature correspondence
Group-wise feature-based alignment
![Page 15: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/15.jpg)
MATCHING ACROSS MODALITIES
Joint Intensity relationship Globally multi-modal Locally linear
MP-RAGE
T2
negative localcorrelation
positive localcorrelation
Non-rigid registration of multi-modal images using both mutual information and cross-correlation.Medical Image Analysis, 2008. A. Andronache, M.V. Siebenthal, G. Szekely, P. Cattin.
15
![Page 16: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/16.jpg)
MATCHING ACROSS MODALITIES
Positive local correlation Conventional correspondence methods,
descriptor matching
MP-RAGE
T2positivecorrelation
16
![Page 17: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/17.jpg)
MATCHING ACROSS MODALITIES
Negative local correlation Conventional correspondence fails Inverted local gradient, orientations ,
descriptor
MP-RAGE
T2
negativecorrelation
21ˆ,ˆ
17
![Page 18: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/18.jpg)
MATCHING ACROSS MODALITIES
Inverted correspondence Rotate orientation, descriptor elements by -π
about Correspondence successful
MP-RAGE
T2
negativecorrelation
3̂
18
![Page 19: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/19.jpg)
MATCHING ACROSS MODALITIES
MP-RAGE, T2, intra-subject No conventional correspondences in GM / WM 22 inverted correspondences within GM / WM
MP-RAGE
T2
19
![Page 20: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/20.jpg)
MATCHING ACROSS MODALITIES
Infant T1 MR: newborn ↔ 2 years old GM / WM contrast inversion due to mylenation No conventional correspondences in GM / WM 4 inverted correspondences within GM / WM
Unbiased average age-appropriate atlases for pediatric studiesNeuroImage 2011. V.S. Fonov, A.C. Evans, D.L. Collins et al.
20
![Page 21: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/21.jpg)
OVERVIEW
Scale-invariant feature representation
Inverted feature correspondence
Group-wise feature-based alignment
![Page 22: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/22.jpg)
GROUP-WISE ALIGNMENT
Automatically align a set of subject images Arbitrary initialization, modalities
22
![Page 23: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/23.jpg)
GROUP-WISE ALIGNMENT: MODEL
ji
iijiji TSIpTp,
)|,(})({
}{ iT
},{ ijij SI
Transform set: image i to atlas (similarity transform)
Feature descriptor, geometry set: image i, feature j
Bayes rule
Conditional feature independence
Marginalization over F
}{ ,lkfF Latent model feature set, feature k,conventional & inverted modes l={0,1}
}){|},({})({}),{|}({ iijijiijiji TSIpTpSITp
ji
lklk
ilkijiji fpTfSIpTp,
,,
, )(),|,(})({
Input features (Iij,Sij) are - Conditionally independent - Identically distributed according to a Gaussian mixture model
23
![Page 24: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/24.jpg)
GAUSSIAN MIXTURE MODEL (GMM)
lk
lklk fpTfSIpTSIp,
,, )(),|,()|,(
)(),|,( 0,30,3 fpTfSIp)(),|,( 0,10,1 fpTfSIp
)(),|,( 0,20,2 fpTfSIp
)(),|,( 0,00,0 fpTfSIp
),( SI
Background24
![Page 25: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/25.jpg)
GMM: UNRECOGNIZED INPUT FEATURE
lk
lklk fpTfSIpTSIp,
,, )(),|,()|,(
)(),|,( 0,30,3 fpTfSIp)(),|,( 0,10,1 fpTfSIp
)(),|,( 0,20,2 fpTfSIp
),( SI
),( SI
)(),|,( 0,00,0 fpTfSIp
Background25
![Page 26: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/26.jpg)
)(),|,( 0,30,3 fpTfSIp
GMM: CONVENTIONAL INPUT FEATURE
lk
lklk fpTfSIpTSIp,
,, )(),|,()|,(
)(),|,( 0,30,3 fpTfSIp)(),|,( 0,10,1 fpTfSIp
)(),|,( 0,20,2 fpTfSIp
),( SI
),( SI
)(),|,( 0,00,0 fpTfSIp
Background26
![Page 27: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/27.jpg)
GMM: INVERTED INPUT FEATURE
lk
lklk fpTfSIpTSIp,
,, )(),|,()|,(
)(),|,( 0,30,3 fpTfSIp)(),|,( 0,10,1 fpTfSIp
)(),|,( 0,20,2 fpTfSIp
),( SI
),( SI
)(),|,( 0,00,0 fpTfSIp
Background
)(),|,( 1,21,2 fpTfSIp
27
![Page 28: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/28.jpg)
LIKELIHOOD: APPEARANCE DESCRIPTOR
lk
lkilkijijiijij fpTfSIpTSIp,
,, )(),|,()|,(
),|()|(),|,( ,,, ilkijlkijilkijij TfSpfIpTfSIp Conditional independence
Descriptor:Isotropic Gaussian density over descriptor elements.
Note: descriptor conditionally independent of alignment Ti,- Fast Matching, no search over Ti -
28
![Page 29: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/29.jpg)
),|(),|(),|()|(),|,( ,,,,, ilkijilkijilkijlkijilkijij TfpTfpTfXpfIpTfSIp
LIKELIHOOD: GEOMETRY
Location: Isotropic Gaussian
lk
lkilkijijiijij fpTfSIpTSIp,
,, )(),|,()|,(
Scale:Gaussian in log σij
Orientation:Isotropic GaussianApproximate Von Mises
29
![Page 30: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/30.jpg)
PRIOR
Latent feature probability: Discrete
lk
lkilkijijiijij fpTfSIpTSIp,
,, )(),|,()|,(
0, lkf
1, lkf
Conventional appearance mode
Inverted appearance mode
30
![Page 31: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/31.jpg)
GROUP-WISE ALIGNMENT: ALGORITHM
Inputs:Volumetric images
Outputs:Alignment solutions: {Ti}
Feature-based model: {fk,l}, p(Iij, Sij| fk,l,Ti), p(fk,l)
31
![Page 32: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/32.jpg)
GROUP-WISE ALIGNMENT: ALGORITHM
1) Feature extraction2) Initialization
Approximate {Ti}
3) Model LearningFixed {Ti}, vary {fk,l}, p(Iij, Sij| fk,l,Ti), p(fk,l)
Mixture model; density, probability parameter estimation.
4) Alignment / Model FittingFixed {fk,l}, p(Iij, Sij| fk,l,Ti), p(fk,l), vary {Ti}
Subject-to-model alignment.
Iterate between 3) & 4) until convergence, i.e. {Ti} no longer changes.
32
![Page 33: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/33.jpg)
1) FEATURE EXTRACTION
Features extracted once from individual images
Note: algorithms use feature data only ~100X data reduction compared to original
image volumes. ~25 seconds for 2563-voxel image, standard PC. ~2K features per brain image
33
![Page 34: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/34.jpg)
2) INITIALIZATION
Approximate image alignment Nearest neighbor descriptor matching Hough transform, similarity
Note: some initial misalignment OK A small subset of image should be aligned Misaligned features sets have negligible impact
34
![Page 35: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/35.jpg)
3) MODEL ESTIMATION
Mixture modeling Estimate set {fk,l}, parameters of p(Iij, Sij| fk,l,Ti), p(fk,l) Robust feature clustering across subjects
Similar to mean-shift algorithm
Note Model conventional appearance Same structure, two distinct latent features, e.g.:
0,1 lkf0,2 lkf
Efficient and robust model-to-image alignment using 3D scale-invariant features.Medical Image Analysis, 2013. Matthew Toews & William M. Wells III.
35
![Page 36: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/36.jpg)
4) MODEL FITTING
Maximum a-posteriori estimation Maximize Ti individually
Conditional independencei
ijijiijiji SITpSITp }),{|(}),{|}({
}),{|( argmax ijijiT
iMAP SITpTi
36
![Page 37: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/37.jpg)
4) MODEL FITTING
Maximum a-posteriori estimation Maximize Ti individually
Two approaches Conventional
Multi-Modal (conventional & inverted modes) Assume
Conditional independencei
ijijiijiji SITpSITp }),{|(}),{|}({
}),{|( argmax ijijiT
iMAP SITpTi
)()( 0,1, lklk fpfp
0)( 1, lkfp
37
![Page 38: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/38.jpg)
EXPERIMENTS
Group-wise Alignment: RIRE data set Modalities: T1, T2, PD, MP-RAGE, CT Brain: 9 subjects, 39 images All subjects exhibit brain tumors
Difficult problem: subject abnormality, no prior information used regarding modalities, initialization.
Compare conventional vs. multi-modal fitting
Comparison and evaluation of retrospective intermodality brain image registration techniquesJournal of Computer Assisted Tomography, 1997. J.B. West, J.M. Fitzpatrick et al.
38
![Page 39: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/39.jpg)
RESULTS: ALIGNMENT
39
![Page 40: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/40.jpg)
RESULTS
Conventional alignment: 3 failure cases (all CT)
Multi-Modal alignment: success
40
Failure case
Success
Failure
![Page 41: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/41.jpg)
RESULTS
Model feature examples
41
![Page 42: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/42.jpg)
DISCUSSION: INVERTED CORRESPONDENCE
Useful for fitting/matching between modalities Less useful once model has been learned May be more prone to false correspondences
Analogous to mutual information Useful when prior information is weak
42
A marginalized MAP approach and EM optimization for pair-wise registrationIPMI 2007. L. Zollei, M. Jenkinson, S. Timoner, W.M. Wells III
![Page 43: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/43.jpg)
DISCUSSION: GROUP-WISE ALIGNMENT
Alignment of difficult multi-modal data Unknown initialization Also effective for infant MR, torso CT, lung CT
Fast 22 minutes (vs. 10-20 hours for group-wise
registration) Deformable alignment?
Global similarity Ti + local linear deformations about correspondences.
43
![Page 44: F EATURE -B ASED A LIGNMENT OF V OLUMETRIC M ULTI - MODAL I MAGES Matthew Toews, Lilla Zöllei, William Wells III June 29, 2013 IPMI.](https://reader030.fdocuments.in/reader030/viewer/2022032600/56649dbf5503460f94ab2dd1/html5/thumbnails/44.jpg)
ACKNOWLEDGEMENTS
NIH grants: P41-EB-015902
P41-RR-013218R00 HD061485-03P41-EB-015898P41-RR-019703
44