Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian....
Transcript of Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian....
![Page 1: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/1.jpg)
Blob detection
![Page 2: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/2.jpg)
Achieving scale covariance• Goal: independently detect corresponding
regions in scaled versions of the same image• Need scale selection mechanism for finding
characteristic region size that is covariant with the image transformationthe image transformation
![Page 3: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/3.jpg)
Recall: Edge detection
fEdge
gdxd Derivative
of Gaussian
d
dx
Edge = maximumg
dxdf ∗
gof derivative
Source: S. Seitz
![Page 4: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/4.jpg)
Edge detection, Take 2
f Edge
gd2
2 Second derivativeof Gaussian
d 2
gdx 2 (Laplacian)
gdxdf 2
2
∗ Edge = zero crossingof second derivative
Source: S. Seitz
![Page 5: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/5.jpg)
From edges to blobs• Edge = ripple• Blob = superposition of two ripples
maximum
Spatial selection: the magnitude of the Laplacianresponse will achieve a maximum at the center ofthe blob, provided the scale of the Laplacian is“matched” to the scale of the blob
![Page 6: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/6.jpg)
Scale selection• We want to find the characteristic scale of the
blob by convolving it with Laplacians at several l d l ki f th iscales and looking for the maximum response
• However, Laplacian response decays as scale increases:increases:
increasing σoriginal signal(radius=8)
Why does this happen?
(radius=8)
![Page 7: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/7.jpg)
Scale normalization• The response of a derivative of Gaussian
filter to a perfect step edge decreases as σiincreases
1πσ 2
1
![Page 8: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/8.jpg)
Scale normalization• The response of a derivative of Gaussian
filter to a perfect step edge decreases as σiincreases
• To keep response the same (scale-invariant), must multiply Gaussian derivative by σmust multiply Gaussian derivative by σ
• Laplacian is the second Gaussian derivative, so it must be multiplied by σ2so it must be multiplied by σ2
![Page 9: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/9.jpg)
Effect of scale normalization
Unnormalized Laplacian responseOriginal signal
Scale-normalized Laplacian response
maximum
![Page 10: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/10.jpg)
Blob detection in 2DLaplacian of Gaussian: Circularly symmetric
operator for blob detection in 2D
222 ggg ∂
+∂
=∇ 22 yxg
∂+
∂=∇
![Page 11: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/11.jpg)
Blob detection in 2DLaplacian of Gaussian: Circularly symmetric
operator for blob detection in 2D
⎟⎟⎞
⎜⎜⎛ ∂
+∂
=∇22
22 ggg σScale-normalized: ⎟⎟⎠
⎜⎜⎝ ∂
+∂
∇ 22norm yxg σScale normalized:
![Page 12: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/12.jpg)
Scale selection• At what scale does the Laplacian achieve a maximum
response to a binary circle of radius r?
r
image Laplacian
![Page 13: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/13.jpg)
Scale selection• At what scale does the Laplacian achieve a maximum
response to a binary circle of radius r?• To get maximum response the zeros of the Laplacian• To get maximum response, the zeros of the Laplacian
have to be aligned with the circle• The Laplacian is given by (up to scale):
• Therefore, the maximum response occurs at
222 2/)(222 )2( σσ yxeyx +−−+.2/r=σ
r 0
image
circle
Laplacian
![Page 14: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/14.jpg)
Characteristic scale• We define the characteristic scale of a blob
as the scale that produces peak of Laplacian i th bl b tresponse in the blob center
characteristic scalecharacteristic scaleT. Lindeberg (1998). "Feature detection with automatic scale selection."International Journal of Computer Vision 30 (2): pp 77--116.
![Page 15: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/15.jpg)
Scale-space blob detector1. Convolve image with scale-normalized
Laplacian at several scales
![Page 16: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/16.jpg)
Scale-space blob detector: Example
![Page 17: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/17.jpg)
Scale-space blob detector: Example
![Page 18: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/18.jpg)
Scale-space blob detector1. Convolve image with scale-normalized
Laplacian at several scales2. Find maxima of squared Laplacian response
in scale-space
![Page 19: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/19.jpg)
Scale-space blob detector: Example
![Page 20: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/20.jpg)
Efficient implementationApproximating the Laplacian with a difference of
Gaussians:
( )2 ( , , ) ( , , )L G x y G x yσ σ σ= +( )( , , ) ( , , )xx yyL G x y G x yσ σ σ+
( ) ( )D G G k G
(Laplacian)
( , , ) ( , , )DoG G x y k G x yσ σ= −(Difference of Gaussians)
![Page 21: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/21.jpg)
Efficient implementation
David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004.
![Page 22: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/22.jpg)
Invariance and covariance properties• Laplacian (blob) response is invariant w.r.t.
rotation and scaling• Blob location and scale is covariant w.r.t.
rotation and scalingWh t b t i t it h ?• What about intensity change?
![Page 23: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/23.jpg)
Achieving affine covariance• Affine transformation approximates viewpoint
changes for roughly planar objects and hl th hiroughly orthographic cameras
![Page 24: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/24.jpg)
Achieving affine covariance
III ⎤⎡⎤⎡ 2 0λ
Consider the second moment matrix of the window containing the blob:
RRIIIIII
yxwMyyx
yxx
yx⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡= −∑
2
112
2
, 00
),(λ
λ
di ti f
direction of the fastest change
Recall: direction of the slowest
change(λmax)-1/2
(λ )-1/2const][ =⎥⎦
⎤⎢⎣
⎡uMvu
Recall:
(λmin) 1/2⎥⎦
⎢⎣v
This ellipse visualizes the “characteristic shape” of the window
![Page 25: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/25.jpg)
Affine adaptation example
Scale-invariant regions (blobs)
![Page 26: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/26.jpg)
Affine adaptation example
Affine-adapted blobs
![Page 27: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/27.jpg)
From covariant detection to invariant description• Geometrically transformed versions of the same
neighborhood will give rise to regions that are related by the same transformationby the same transformation
• What to do if we want to compare the appearance of these image regions?
• Normalization: transform these regions into same-size circles
![Page 28: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/28.jpg)
Affine normalization• Problem: There is no unique transformation from an
ellipse to a unit circle• We can rotate or flip a unit circle and it still stays a unit circleWe can rotate or flip a unit circle, and it still stays a unit circle
![Page 29: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/29.jpg)
Eliminating rotation ambiguity• To assign a unique orientation to circular
image windows:C t hi t f l l di t di ti i th t h• Create histogram of local gradient directions in the patch
• Assign canonical orientation at peak of smoothed histogram
0 2 π
![Page 30: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/30.jpg)
From covariant regions to invariant features
Eliminate rotational Compute appearanceExtract affine regions Normalize regions
Eliminate rotational ambiguity
Compute appearancedescriptors
SIFT (Lowe ’04)SIFT (Lowe 04)
![Page 31: Blob detection - Computer Sciencelazebnik/spring11/lec08_blob.pdfimage circle Laplacian. Characteristic scale • We define the characteristic scale of a blob as the scale that produces](https://reader034.fdocuments.in/reader034/viewer/2022050314/5f760e054443822e6e582389/html5/thumbnails/31.jpg)
Invariance vs. covarianceInvariance:
• features(transform(image)) = features(image)
Covariance:• features(transform(image)) = transform(features(image))
Covariant detection => invariant description