KECE471 Computer Vision Pyramidal Image...
Transcript of KECE471 Computer Vision Pyramidal Image...
![Page 1: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/1.jpg)
KECE471 Computer Vision
Pyramidal Image
Representation
Chang-Su Kim
Sections 7.7 and 9.2, Computer Vision by Forsyth and PonceNote: Most contents were extracted from the lecture notes of Prof. Kyoung Mu Lee.
![Page 2: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/2.jpg)
A curve corresponds to • a hair on the nose in the
biggest image• a stripe in the medium size
image• nose itself in the smallest
image
Image Pyramid: Example
![Page 3: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/3.jpg)
![Page 4: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/4.jpg)
Pyramidal Representation
• Pyramidal representation is a kind of scaled
representation
• Both large and small scaled information are
interesting
– Big bars and small bars
– Stripes and hairs
![Page 5: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/5.jpg)
Fine (high) resolution
Coarse (low) resolution
Image Pyramid (it is not an Egyptian tomb)
![Page 6: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/6.jpg)
Aliasing
• Lowpass filtering is required before downsampling to
avoid aliasing
• Anti-aliasing filtering
• A Gaussian filter is often used
Without anti-aliasing filtering
![Page 7: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/7.jpg)
Aliasing
• Lowpass filtering is required before subsampling to
avoid aliasing
• Anti-aliasing filtering
• A Gaussian filter is often used
With anti-aliasing filtering
![Page 8: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/8.jpg)
Aliasing Anti-aliased
![Page 9: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/9.jpg)
Gaussian Pyramid
Gaussian filtering
Downsampling
2)*( 23 gaussianGG
Image0G
2)*( 12 gaussianGG
2)*( 34 gaussianGG
1 0( * ) 2G G gaussian
![Page 10: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/10.jpg)
http://web.mit.edu/persci/people/adelson/pub_pdfs/pyramid83.pdf
filter mask
G 1/4
G 1/8
Gaussian 1/2
Construction of a Gaussian Pyramid
![Page 11: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/11.jpg)
Applications of Gaussian Pyramids
• Search for correspondence
– look at coarse scales, then refine with finer
scales
• Edge tracking
– a “good” edge at a fine scale has parents at a
coarser scale
• Template matching
– e.g. Detecting faces
![Page 12: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/12.jpg)
Hierarchical Block Matching
• Lower resolution motion vector is used to predict higher resolution motion vector (e.g. d2,0,1 is used to predict d3,1,2)
– Reduction of computational complexity
– More reliable motion vector estimation
~
~
![Page 13: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/13.jpg)
𝐝1 = (3,3)
𝐝2 = 2 × 𝐝1 = 6,6
𝐪2 = 1,−1
𝐝2 = 𝐝2 + 𝐪2 = (7,5)
𝐝3 = 2 × 𝐝2 = 14,10
𝐪3 = −1, 1
𝐝3 = 𝐝3 + 𝐪3 = (13,11)
![Page 14: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/14.jpg)
Non-Hierarchical Block Matching Algorithm
![Page 15: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/15.jpg)
Hierarchical Block Matching Algorithm
![Page 16: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/16.jpg)
Template Matching
• Strategy 1
– Use templates of
different sizes
– For large
templates,
matching is costly
![Page 17: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/17.jpg)
Template Matching
• Strategy 2
– Apply a fixed-size
template to the
Gaussian pyramid
![Page 18: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/18.jpg)
Multiscale Saliency Detection Using
Random
Walk with Restart
Jun-Seong Kim, Jae-Young Sim, and Chang-Su Kim
To appear in IEEE Trans. Circuits Syst. Video Technol., 2013
![Page 19: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/19.jpg)
Feature Extraction
![Page 20: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/20.jpg)
Random Walk
• Edge weight 𝑤𝑖𝑗
– Feature difference between nodes 𝑖 and 𝑗
• Equilibrium state
𝑃𝜋 = 𝜋
![Page 21: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/21.jpg)
Random Walk
![Page 22: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/22.jpg)
Scales
![Page 23: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/23.jpg)
Hierarchical Saliency Refinement
• Equilibrium state
𝐫fine = 1 − 𝜖 𝐏𝐫fine + 𝜖𝑈(𝐫coarse)
![Page 24: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/24.jpg)
![Page 25: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/25.jpg)
Laplacian Pyramid
• It removes redundancies in Gaussian
Pyramid
• Similar to edge images
• Most pixels are zero
• It can be used in point detection and
image compression
![Page 26: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/26.jpg)
Laplacian Pyramid
• Gaussian Pyramid
– G0
– G1=D(G0)
– G2=D(G1)
– G3=D(G2)
• D
– Gaussian filtering
– then
Downsampling
• Laplacian Pyramid
– L0=G0-U(G1)
– L1=G1-U(G2)
– L2=G2-U(G3)
– L3=G3
• U
– Upsampling
![Page 27: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/27.jpg)
Laplacian Pyramid
• Gaussian Pyramid
– G0
– G1=D(G0)
– G2=D(G1)
– G3=D(G2)
• Laplacian Pyramid
– L0=G0-U(G1)
– L1=G1-U(G2)
– L2=G2-U(G3)
– L3=G3
![Page 28: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/28.jpg)
Laplacian Pyramid
• Analysis
– L0=G0-U(G1)
– L1=G1-U(G2)
– L2=G2-U(G3)
– L3=G3
• Synthesis
– G0=L0+U(G1)
– G1=L1+U(G2)
– G2=L2+U(G3)
– G3=L3
![Page 29: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/29.jpg)
Gaussian Pyramid
![Page 30: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/30.jpg)
Laplacian Pyramid
![Page 31: KECE471 Computer Vision Pyramidal Image Representationmcl.korea.ac.kr/.../05/09_Pyramidal-Image-Representation.pdf · 2017. 5. 10. · Pyramidal Image Representation Chang-Su Kim](https://reader036.fdocuments.in/reader036/viewer/2022081623/6140eae783382e045471c1c6/html5/thumbnails/31.jpg)
Laplacian Pyramid for Compression