Geometric Transformations(based on Joan Serrat slides)
Felipe LumbrerasDept. Ciències de la Computació / Centre de Visió per Computador
Universitat Autònoma de Barcelonahttp://www.cvc.uab.es/shared/teach/a102784/
Geometric Transformations
• Introduction
• Mapping computation– Scene Sensor modelling
– Control points
– Semi-automatic control point determination
– Automatic control point determination
• Resampling
• Applications– Registration
– Bird’s view
– Mosaic of images
Introduction
Image transforms according to their formulation :
• point
• local neighbourhood
Introduction
Image transforms according to their formulation :
• global
• arithmetic (+,-,*,/)
u
v
Introduction
Image transforms according to their formulation :
• geometric
transform coordinates, not pixel values
Introduction
Main applications:
• Correct image distortions caused by acquisition devices or geometry
• Compare images
Fields:
• Remote sensing
• Medical imaging
• …
Introduction
Camera distortions
Introduction
Satellite images
Introduction
Satellite images
Orthorectified image:
each pixel “seen
from above”
Introduction
Satellite images
Sensor fusion
IKONOS4m color
1m panchromatic fusion(pansharpening)
+ =
Introduction
Medical Imaging
Frame registration on eye angiographies
original registered
Introduction
Medical Imaging
Frame registration on eye angiographies
original registered
Introduction
Medical Imaging
Frame registration on eye angiographies
original registered
Introduction
A geometric transform has two steps:
1. Computation of the mappingFind the equation relating the coordinates systems of original and result images.
2. Resampling or interpolationAssign values to the pixels of the result image from those of the input image.
Mapping
Introduction (Output to input)
Input to output :problem?
Output to input
Introduction
Zoom
input to output output to input
input
outputs
Introduction
Rotation
input to output output to input
input
outputs
Mapping computation
• Model mathematically the acquisition process and invert it– Example: earth rotation,
effect in satellite images
– Example: rolling shutter
in photogrammetry
𝑥 = 𝑘 𝑅 −𝑅𝑐 𝑋 𝑥 = 𝑘 𝑅 −𝑅(𝑡)𝑐(𝑡) 𝑋rolling shutter
modelglobal shutter
model
Mapping computation
• Adjustment to control points
Mapping computation
• Adjustment to control points
input data: pairs of corresponding control points
𝑘 = 𝑞1 + 𝑞2𝑖 + 𝑞3𝑗 + 𝑞4𝑖2 + 𝑞5𝑖𝑗 + 𝑞6𝑗
2 +⋯
𝑙 = 𝑟1 + 𝑟2𝑖 + 𝑟3𝑗 + 𝑟4𝑖2 + 𝑟5𝑖𝑗 + 𝑟6𝑗
2 +⋯
𝑖𝑚, 𝑗𝑚 , 𝑘𝑚, 𝑙𝑚 , 𝑚 = 1…𝑛
Mapping computation• Adjustment to control points
• Matrix notation
𝑄 𝑖, 𝑗 = 𝑘 = 𝑞1 + 𝑞2𝑖 + 𝑞3𝑗 + 𝑞4𝑖2 + 𝑞5𝑖𝑗 + 𝑞6𝑗
2 +⋯
𝑅 𝑖, 𝑗 = 𝑙 = 𝑟1 + 𝑟2𝑖 + 𝑟3𝑗 + 𝑟4𝑖2 + 𝑟5𝑖𝑗 + 𝑟6𝑗
2 +⋯
min𝑟1…𝑟𝑝
𝑚=1
𝑛
𝑙𝑚 − 𝑅 𝑖𝑚, 𝑗𝑚2
min𝑞1…𝑞𝑝
𝑚=1
𝑛
𝑘𝑚 − 𝑄 𝑖𝑚, 𝑗𝑚2
𝐾 = [𝑘1, 𝑘2, … 𝑘𝑛]𝑇, 𝐿 = [𝑙1, 𝑙2, … 𝑙𝑛]
𝑇 , 𝑄 = [𝑞1, 𝑞2, … 𝑞𝑛]𝑇, 𝑅 = [𝑟1, 𝑟2, … 𝑟𝑛]
𝑇
M=
1 𝑖1 𝑗1 𝑖12 𝑖1𝑗1 𝑗1
2 𝑖13 ⋯
1 𝑖2 𝑗2 𝑖22 𝑖2𝑗2 𝑗2
2 𝑖23 ⋯
⋮⋮
1 𝑖𝑛 𝑗𝑛 𝑖𝑛2 𝑖𝑛𝑗𝑛 𝑗𝑛
2 𝑖𝑛3 ⋯
𝐾 = M𝑄 𝐿 = M𝑅
𝑄 = M𝑇M −1M𝑇𝐾
𝑅 = M𝑇M −1M𝑇𝐿
Solution using pseudoinverse matrix
Mapping computation
• Adjustment to control points
Manually selecting a number of corresponding control points is:
– time consuming
– tedious
– error prone
– imprecise
Mapping computation
• Adjustment to control points
Semi-automatically: given control points on the input image, find the corresponding points on the reference (output)
Similarity measure: correlation, covariance, normalized cross-correlation
Mapping computation
• Adjustment to control points
Automatically: find a set of local features in both images and look for a transformation in a robust way (RANSAC: RANdom SAmple Consensus)
RANSAC
• RANdom SAmple Consensus: iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers.
1.- Select randomly the minimum number of points required to determine the model parameters. Solve for the parameters of the model.
2.- Determine how many points from the set of all points fit with a predefined tolerance (inliers).
3.- If inliers exceeds a predefined threshold τ, re-estimate the model parameters using all the identified inliers and terminate.
4.- Else repeat steps 1 through 3 a maximum of N times.
source: http://en.wikipedia.org/wiki/RANSAC
Resampling
Resampling
Given a certain mapping, fill in the output image by interpolating the pixel values of the input image at non integer coordinates
Resampling
• Zero order interpolation: nearest neighbour
Resampling
• First order interpolation: in 2D images bilinear
Resampling
• Third order interpolation: in 2D images bicubic
source: https://en.wikipedia.org/wiki/Bicubic_interpolation
Resampling
• Ideal interpolation for bandlimited signal sampled at Nyquist rate: convolution with sinc 𝑥, 𝑦
sinc 𝑥 =sin(𝑥)
𝑥
Nyquist rate: is the minimum rate at which a finite bandwidth (bandlimited) signal needs to be sampled to retain all of the information.
For a bandwidth of span B, Nyquist frequency = 2 B. If a signal is sampled at regular intervals dt, Nyquist rate = 1/(2dt)
sinc 𝑥, 𝑦
Resampling
• Comparison example180o rotation by reflection (no interpolation needed) versus 36 successive rotations of 5o each
. . .
5° 10° 15°
170° 175° 180°
original
Resampling
• Comparison example3 input images
Resampling
• Biliniar interpolation (2 x 2 pixels)
• Bicubic (4 x 4 pixels)
Resampling
• Lanczos kernel (6 x 6 pixels)
• sinc (x,y) (16 x 16 pixels)
Applications
Applications
• Registration
• Geometric alignment of images in order to compare or fusion pixel
by pixel.
• More used in medical imaging and remote sensing.
• Detects temporal differences or combine different sensors.
• Projections:
similarities, affinities, projective, spherical, cylindrical, warpings
Applications
• Registration : Geometric alignment for 3D fusion CT-MRI
CT MRI
Applications
• Registration : Geometric alignment
Sagital slicesAxial slices
MRI volume
Applications
• Registration :
Geometric alignment
Applications
• Registration : Alignment to stabilize a video sequence
Original sequence Stabilized sequence
Applications
• Registration :
Alignment to stabilize a
video sequence
A arteryV vein
Applications
• Construction of stitching mosaics from individual image
– Mathematical model of a camera: pin-hole, central projection.
– What a homography is?
– Mosaics of stitching images.
Applications
• Mathematical model of a camera: pin-hole, central
projection.
Applications
• Mathematical model of a camera: pin-hole, central
projection.
focal distance
holeimage
plane
Applications
• Mathematical model of a camera: pin-hole, central
projection.
focal distance
variation
Applications
• Mathematical model of a camera: pin-hole, central
projection.
photographic paper (negative)
1 2 3 4
Applications
• Mathematical model of a camera: pin-hole, central projection.
Applications
• What a homography is?
A particular case: central projection of a plane into a another plane
(2D to 2D).
There are other projections: orthographic, weak-perspective
Applications
• What a homography is?
Examples of interest: changing the point of view in the image of a plane.
Applications
• What a homography is?
Examples of interest: changing the point of view
in the image of a plane.
Applications
• Panoramic mosaics
Applications
projecció rectilínia (homografia) 100°
Panoramics of Georges Lagarde (GURL)
Applications
projecció rectilínia (homografia) 130°
Panoramics of Georges Lagarde (GURL)
Applications
projecció rectilínia (homografia) 170°
Panoramics of Georges Lagarde (GURL)
Applications
Panoramics of Georges Lagarde (GURL)
Applications
Panoramics of Georges Lagarde (GURL)
Impossible ?
Applications
Panoramics of Georges Lagarde (GURL)
Applications
• Lane road detection
Applications
• Lane road detection
Applications
• Lane road detection
Applications
• Lane road detection
Applications
• Lane road detection
s
−60°
+60°
Radon transform
θ
s
θi
g(s,θi)
Applications
• Lane road detection
Applications
• Distances in a plane
Applications
• Distances in a plane
Mapas obtenido mediante imágenes aéreasTFG: Rubén Pérez Conte (2014)
Top Related