Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image...
-
Upload
justin-watkins -
Category
Documents
-
view
218 -
download
0
Transcript of Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image...
![Page 1: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/1.jpg)
Image Registration as
an Optimization Problem
![Page 2: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/2.jpg)
Overlaying two or more images of the same scene
Image Registration
![Page 3: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/3.jpg)
Image Registration
![Page 4: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/4.jpg)
IMAGE REGISTRATION METHODOLOGY
1. Control point selection
2. Control point matching
3. Transform model estimation
4. Image resampling
and transformation
Four basic steps of image registration
![Page 5: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/5.jpg)
CONTROL POINT SELECTION
- distinctive points
- corners
- lines
- closed-boundary regions
- virtual invariant regions
- window centers
![Page 6: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/6.jpg)
Optimization problem:
Finding local extremes of a “cornerness” function
CORNER DETECTION
![Page 7: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/7.jpg)
Optimization method: usually full search with constraints
Cornerness functions use to have many extremes
CORNER DETECTION
![Page 8: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/8.jpg)
Corner detectors
Kitchen & Rosenfeld Harris Non-dif
![Page 9: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/9.jpg)
2. Control point matching
![Page 10: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/10.jpg)
Control point matching
Signal-based methodsSimilarity measures calculated directly from the image graylevelsExamples - Image correlation, image differences, phase correlation, mutual information, …
Feature-based methodsSymbolic description of the featuresMatching in the feature space (classification)
![Page 11: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/11.jpg)
Image correlation
W
I
(k,m)
C(k,m) = ( Ik,m - mean ( Ik,m )) . ( W - mean ( W ))
( Ik,m - mean ( Ik,m )) 2. ( W - mean ( W )) 2
![Page 12: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/12.jpg)
Image correlation
W
I
(k,m)
max C(k,m) – full search if W is small
- gradient-based methods for large
W
- the image must be spatially
correlated
![Page 13: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/13.jpg)
Mutual information method
W
I
Statistical measure of the
dependence between
two images
MI(f,g) = H(f) + H(g) – H(f,g)
Often used for multimodal
image registration
Popular in medical imaging
![Page 14: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/14.jpg)
I (X;Y ) = H (X ) + H (Y ) – H (X,Y )
Entropy
Joint entropy
Mutual infomation
MUTUAL INFORMATION
H(X) = - Σ p(x) log p(x)x
H(X,Y) = - Σ Σ p(x,y) log p(x,y)x y
![Page 15: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/15.jpg)
• Very often applied to 3D volume data
• Rotation estimation may be also required
• Good initial guess is available
• MI calculation is very time-consuming
• Full search is not feasible
• Sophisticated optimization algorithms
• Powell’s optimization method
MUTUAL INFORMATION
![Page 16: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/16.jpg)
The cab method Powell’s method
Powell’s optimization method
![Page 17: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/17.jpg)
![Page 18: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/18.jpg)
Refinement of control point location
• Mutual information method
![Page 19: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/19.jpg)
Pyramidal representationProcessing from coarse to fine
level
![Page 20: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/20.jpg)
Feature-based methods
• Combinatorial matching (no feature description). Graph matching, parameter clustering. Global information only is used.
• Matching in the feature space (pattern classification). Local information only is used.
• Hybrid matching (combination of both to get higher robustness)
![Page 21: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/21.jpg)
Matching in the feature space
( v11, v21, v31, …
)
( v12, v22, v32, …
)
min distance(( v1k, v2k, v3k, … ) , ( v1m, v2m, v3m, … ))k,m
![Page 22: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/22.jpg)
Mapping function design
• Global functions
Similarity, affine, projective transform
Low-order polynomials
![Page 23: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/23.jpg)
Similarity transform – Least square fittranslation [ x, y], rotation , uniform scaling s
x’ = s (x cos - y sin ) + xy’ = s (x sin + y cos ) + y
s cos = a, s sin = b
min ( i=1 {[ xi’– (axi - byi ) - x ]2+[ yi’ – (bxi + ayi ) - y ]2})
(xi2 + yi
2) 0 xi yi a (xi
’xi - yi
’ yi ) 0 (xi
2 + yi2) - yi xi b
(yi’xi
- xi’ yi )
xi - yi
N 0 x = xi’
yi xi
0 N y yi’
![Page 24: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/24.jpg)
Transform models
• Global functions
Similarity, affine, projective transform
Low-order polynomials
• Local functions
Piecewise affine, piecewise cubic
Thin-plate splines
Radial basis functions
![Page 25: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/25.jpg)
A motivation to TPS
• Unconstrained interpolation – ill posed• Constrained interpolation
min
This task has an analytical solution - TPS
![Page 26: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/26.jpg)
TPS
![Page 27: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/27.jpg)
Computing the TPS coefficients
![Page 28: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/28.jpg)
Approximating TPS
• Regularized approximation – well posed
min J(f) = E(f) + b R(f)
E(f) - error term R(f) - regularization term
b - regularization parameter
![Page 29: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/29.jpg)
The choice of E and R
E(f) = (xi’ – f(xi,yi))2
R(f) =
The solution of the same form - “smoothing” TPS
![Page 30: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/30.jpg)
Computing the smoothing TPS coefficients
![Page 31: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/31.jpg)
The role of parameter b
![Page 32: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/32.jpg)
TPS, 1 >> b
![Page 33: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/33.jpg)
TPS, 1 > b
![Page 34: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/34.jpg)
TPS, 1 < b
![Page 35: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/35.jpg)
TPS, 1 << b
![Page 36: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/36.jpg)
The original
![Page 37: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/37.jpg)
The choice of “optimal” parameter b
• “Leave-one-out” cross-validation
• Minimizing the mean square error over b
• Possibly unstable
![Page 38: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/38.jpg)
TPS registrationdeformed reference
![Page 39: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/39.jpg)
3D shape recovery by TPS
![Page 40: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/40.jpg)
The drawback: evaluation of TPS is slow
Speed-up techniques
• Adaptive piecewise approximation
• Subtabulation schemes (Powell)
• Approximation by power series
![Page 41: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/41.jpg)
Adaptive piecewise approximation
• Image decomposition according to the distortion (quadtree, bintree, ... )
• Transformation of each block by affine or projective transform
![Page 42: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/42.jpg)
Adaptive piecewise approximation
TPS
![Page 43: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/43.jpg)
Adaptive piecewise approximation
Piecewise
projective
![Page 44: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/44.jpg)
Adaptive piecewise approximation
![Page 45: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/45.jpg)
Subtabulation schemes
• Calculate the TPS values on a coarse grid
• Refine the grid twice
• Approximate the missing TPS values
![Page 46: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/46.jpg)
Powell’s scheme
![Page 47: Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649f045503460f94c180e8/html5/thumbnails/47.jpg)
Luner’s scheme