Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf ·...
Transcript of Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf ·...
![Page 1: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/1.jpg)
Computer Vision I -Algorithms and Applications:
Basics of Digital Image Processing – Part 2
Carsten Rother
Computer Vision I: Basics of Image Processing
06/11/2013
![Page 2: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/2.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 2
![Page 3: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/3.jpg)
Reminder: Convolution
06/11/2013Computer Vision I: Basics of Image Processing 3
• Replace each pixel by a linear combination of its neighbours and itself.
• 2D convolution (discrete)
𝑔 = 𝑓 ∗ ℎ
𝑔 𝑥, 𝑦 = 𝑘,𝑙 𝑓 𝑥 − 𝑘, 𝑦 − 𝑙 ℎ 𝑘, 𝑙 = 𝑘,𝑙 𝑓 𝑘, 𝑙 ℎ 𝑥 − 𝑘, 𝑦 − 𝑙
𝑓 𝑥, 𝑦 ℎ 𝑥, 𝑦 g 𝑥, 𝑦
Centred at 0,0
![Page 4: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/4.jpg)
Reminder – Linear Filters
06/11/2013Computer Vision I: Basics of Image Processing 4
![Page 5: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/5.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 5
![Page 6: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/6.jpg)
Gaussian Image Pyramid
06/11/2013Computer Vision I: Basics of Image Processing 6
• Represent Image at multiple resolution
High resolution Low resolution
![Page 7: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/7.jpg)
A naive approach
06/11/2013Computer Vision I: Basics of Image Processing 7
[From book: Computer Vision A modern Approach, Ponce and Forsyth]
Take every second pixel – bad!
![Page 8: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/8.jpg)
Problem: Aliasing Effect
06/11/2013Computer Vision I: Basics of Image Processing 8
Problem: High frequencies (sharp transitions) are lost
![Page 9: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/9.jpg)
Solution: Smooth before downsampling
06/11/2013Computer Vision I: Basics of Image Processing 9
![Page 10: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/10.jpg)
Application: template search
06/11/2013Computer Vision I: Basics of Image Processing 10
Search template:
[from Irani, Basri]
![Page 11: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/11.jpg)
Application: Large Image Segmentation
06/11/2013Computer Vision I: Basics of Image Processing 11
Banded Segmentation
Small image(100x100)
Small segmentation result
Large image, e.g. 10 MPixel Trimap: created from small image
Segmentation large image
![Page 12: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/12.jpg)
Application: Large Label Space
06/11/2013Computer Vision I: Basics of Image Processing 12
Approach: 1. solve problem on small image (𝑥5 downscale)
with 40 𝑥 40 label space (coarse motion)(1600 labels)
2. Do on full resolution only in 5𝑥5 neighbourhood around each solution (add fine motion)
(25 labels)
Color coding2 images (overlaid)
Motion 200 𝑥 200 possible
discrete movements(40.000 labels)
(problem small objects can get lost)
![Page 13: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/13.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 13
![Page 14: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/14.jpg)
Goal: Find long edge chains
06/11/2013Computer Vision I: Basics of Image Processing 14
What do we want:
• Good detection: we want to find edges not noise
• Good localization: find true edge• Single response: one per edge
(independent of edge sharpness)
• Long edge-chains
![Page 15: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/15.jpg)
Idealized edge types
06/11/2013Computer Vision I: Basics of Image Processing 15
We focus on this
![Page 16: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/16.jpg)
What are edges ?
• correspond to fast changes in the image
• The magnitude of the derivative is large
06/11/2013Computer Vision I: Basics of Image Processing 16
Image of 2 step edges
Slizce through the image
Image of 2 ramp edges
Slizce through the image
![Page 17: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/17.jpg)
What are fast changes in the image?
06/11/2013Computer Vision I: Basics of Image Processing 17
Image
Scanline 250
Scanline 250 smoothed with Gausian
Texture or many edges?
Edges defined after smoothing
![Page 18: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/18.jpg)
Edges and Derivatives
06/11/2013Computer Vision I: Basics of Image Processing 18
We will look at this first
![Page 19: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/19.jpg)
Edge filters in 1D
06/11/2013Computer Vision I: Basics of Image Processing 19
We can implement this as a linear filter:
Forward differences:
Central differences: -1 0 11/2
1/2 -1 1
![Page 20: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/20.jpg)
Reminder: Seperable Filters
06/11/2013Computer Vision I: Basics of Image Processing 20
This is the centralized differencesoperator
![Page 21: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/21.jpg)
Edge Filter in 1D: Example
06/11/2013Computer Vision I: Basics of Image Processing 21
Based on 1st derivative• Smooth with Gaussian – to filter out noise• Calculate derivative• Find its optima
![Page 22: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/22.jpg)
Edge Filtering in 1D
06/11/2013Computer Vision I: Basics of Image Processing 22
Simplification: (saves one operation)
Derivative of Gaussian
![Page 23: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/23.jpg)
Edge Filtering in 2D
06/11/2013Computer Vision I: Basics of Image Processing 23
![Page 24: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/24.jpg)
Edge Filter in 2D: Example
06/11/2013Computer Vision I: Basics of Image Processing 24
![Page 25: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/25.jpg)
Edge Filter in 2D
06/11/2013Computer Vision I: Basics of Image Processing 25
x-derivatives with different Gaussian smoothing
![Page 26: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/26.jpg)
What is a gradient
06/11/2013Computer Vision I: Basics of Image Processing 26
![Page 27: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/27.jpg)
What is a gradient
06/11/2013Computer Vision I: Basics of Image Processing 27
![Page 28: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/28.jpg)
What is a gradient
06/11/2013Computer Vision I: Basics of Image Processing 28
![Page 29: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/29.jpg)
What is a Gradient
06/11/2013Computer Vision I: Basics of Image Processing 29
![Page 30: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/30.jpg)
Example – Gradient magnitude
06/11/2013Computer Vision I: Basics of Image Processing 30
Our goal was to get thin edge chains?
First smoothed with Gaussian First smoothed with broad Gaussian
![Page 31: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/31.jpg)
How to get edge chains
06/11/2013Computer Vision I: Basics of Image Processing 31
1. Compute robust Gradient Image: ( (𝐷𝑥 ∗ 𝐺) ∗ 𝐼 , (𝐷𝑦 ∗ 𝐺) ∗ 𝐼)
2. Find edge-points (“edgels”): non-maximum suppression
3. Link-up edge-points to get chains
4. Do hysteresis to clean-up chains
Rough Outline of a good edge detector (such as Canny)
edge-points or edgel
![Page 32: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/32.jpg)
Non-maximum surpression
06/11/2013Computer Vision I: Basics of Image Processing 32
1. Check if pixel is local maximum at gradient orientation (interpolate values for p,r)
2. Accept edge-point if above a threshold
Wich pixel is an edge point ? (non-max surpression)
Edge point
Not Edge point
![Page 33: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/33.jpg)
Link up edge-points to get chains
06/11/2013Computer Vision I: Basics of Image Processing 33
1. Link-up neighbouring pixels if both are edge-points.
Edge point
Not Edge point
![Page 34: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/34.jpg)
Clean up chains with Hysteresis
06/11/2013Computer Vision I: Basics of Image Processing 34
High start threshold
Low threshold along the chain
Keep a chain:
![Page 35: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/35.jpg)
Final Result
06/11/2013Computer Vision I: Basics of Image Processing 35
Image Not much smoothing (fine scale)
much smoothing (coarse scale)small threshold
much smoothing (coarse scale)large threshold
![Page 36: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/36.jpg)
Alterative: Edge detection with Laplacian Filter
06/11/2013Computer Vision I: Basics of Image Processing 36
Called Laplacian of Gaussian (LoG)
![Page 37: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/37.jpg)
Laplacian example in 1D
06/11/2013Computer Vision I: Basics of Image Processing 37
Find zero-crossing
![Page 38: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/38.jpg)
Approximate LoG with Difference of Gaussian (DoG)
06/11/2013Computer Vision I: Basics of Image Processing 38
Solid Line: Differnce of Gaussian (DoG)
Dashed Line:Laplacian of Gaussian
![Page 39: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/39.jpg)
Laplacian example in 2D
06/11/2013Computer Vision I: Basics of Image Processing 39
![Page 40: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/40.jpg)
Future Lecture: Segmentation
06/11/2013Computer Vision I: Basics of Image Processing 40
• So far we looked at “jumps” in gray-scale images?
• Humans perceive edges very differently (edges depend on semantic)
“average human drawing”
[from Martin, Fowlkes and Mail 2004]
Hard for computer vision method which operates only locally!
![Page 41: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/41.jpg)
Try to learn semantically meaningful image edges
06/11/2013Computer Vision I: Basics of Image Processing 41
• Features: brightness gradient; color gradient; texture gradient; weighted combination, etc.
![Page 42: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/42.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 42
![Page 43: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/43.jpg)
Hough voting with edgels (edge-points)
06/11/2013Computer Vision I: Basics of Image Processing 43
Hough Space(line is now a point)
Algorithm:
1. Empty cells in Hough Space2. Put for each Edgel(𝜃, 𝑟) into a cell of the Hough Space 3. Find Peaks in Hough Space (use non-max suppression)4. Re-fit all edgels to a single line
Hough transform
3 edgels(edge-points with direction)
![Page 44: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/44.jpg)
Hough Voting: original
06/11/2013Computer Vision I: Basics of Image Processing 44
Goal: find all points with many “votes” in accumulator space
Hough transform
Goal: find all lines
Image with just 3 pointsAll lines that go through the 3 points
This idea of transformation to a voting space can be used for many scenarios
Image with points
![Page 45: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/45.jpg)
Hough transform: original
06/11/2013Computer Vision I: Basics of Image Processing 45
[From Wikipedia]
All possiblelines at each point
![Page 46: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/46.jpg)
Example: Orthogonal Vanishing point detection
06/11/2013Computer Vision I: Basics of Image Processing 46
[Rother, Thesis ‘03]
846 line segments found
Found 3 orthogonal vanishing points
Algorithm: RANSAC (explained later)Application: Camera Calibration (see later)
![Page 47: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/47.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 47
![Page 48: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/48.jpg)
What region should we try to match?
06/11/2013Computer Vision I: Basics of Image Processing 48
Look for a region that is unique, i.e. not ambiguous
We want to find a few regions where this image pair matches: Applications later
![Page 49: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/49.jpg)
Goal: Interest Point Detection
• Goal: predict a few “interest points” in order to remove redundant data efficiently
06/11/2013Computer Vision I: Basics of Image Processing 49
• Should be invariant against:
a. Geometric transformation – scaling, rotation, translation, affine transformation, projective transformation etc.
b. Color transformation – additive (lightning change), multiplicative (contrast), linear (both), monotone etc.;
c. Discretization (e.g. spatial resolution, focus);
(scetch)
![Page 50: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/50.jpg)
Points versus Lines
06/11/2013Computer Vision I: Basics of Image Processing 50
„Apeture problem“
Lines are not as good as points
?
![Page 51: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/51.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 51
[Szeliski and Seitz]
Local measure of feature uniqueness: Shifting the window in any direction: how does it change
Shift left Shift top,left
![Page 52: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/52.jpg)
06/11/2013Computer Vision I: Basics of Image Processing 52
Harris Detector
How similar is the image to itself?
Autocorrelation function:
is a small vicinity (window) around
is a convolution kernel, used to decrease the influence of pixels far from , e.g. the GaussianFor simplicity we use 𝑤(𝑢, 𝑣) = 1
𝑥
𝑦
Δ𝑥, Δ𝑦
![Page 53: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/53.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 53
One is interested in properties of at each position
Let us look at a linear approximation of Taylor expansion around
+ 𝜖(Δ𝑥, Δ𝑦)
Gradient at (𝑢, 𝑣)
![Page 54: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/54.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 54
Put it together:
with
Q: Structure Tensor
We compute this at any image location (𝑥, 𝑦)
![Page 55: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/55.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 55
The autocorrelation function
Function c is (after approximation) a quadratic function in and
• Isolines are ellipses ( is symmetric and positive definite);• Eigenvector 𝑥1 with (larger) Eigenvalue 𝜆1 is the direction of fastest change in
function 𝑐• Eigenvector 𝑥2 with (smaller) Eigenvalue 𝜆2 is direction of slowest change in
function 𝑐
Δ𝑥
Δy
Function c
𝑥2
𝑥1
Note 𝑐 = 0 for Δ𝑥 = Δy = 0
![Page 56: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/56.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 56
Some examples – isolines for :
(a) Flat (b) Edges (c) Corners
a. Homogenous regions: both -s are small
b. Edges: one is small the other one is large
c. Corners: both -s are large (this is what we are looking for!)
![Page 57: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/57.jpg)
Harris Detector
06/11/2013Computer Vision I: Basics of Image Processing 57
Zoom in
smaller eigenvaluelarger eigenvalue𝜆1 𝜆2Image
![Page 58: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/58.jpg)
Harris detector
06/11/2013Computer Vision I: Basics of Image Processing 58
“Cornerness” is a characteristic of
Proposition by Harris:
Downweights edges where 𝜆1 ≫ 𝜆2
Smallest eigenvalueHarris Value
![Page 59: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/59.jpg)
Harris Corners - example
06/11/2013Computer Vision I: Basics of Image Processing 59
![Page 60: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/60.jpg)
h-score (red- high, blue - low)
06/11/2013Computer Vision I: Basics of Image Processing 60
![Page 61: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/61.jpg)
Threshold (H-score > value)
06/11/2013Computer Vision I: Basics of Image Processing 61
![Page 62: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/62.jpg)
Non-maximum suppression
06/11/2013Computer Vision I: Basics of Image Processing 62
![Page 63: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/63.jpg)
Harris corners in red
06/11/2013Computer Vision: Algorithms and Applications --
- Carsten Rother63
![Page 64: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/64.jpg)
Other examples
06/11/2013Computer Vision I: Basics of Image Processing 64
![Page 65: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/65.jpg)
Maximally stable extremal regions
06/11/2013Computer Vision I: Basics of Image Processing 65
• Invariant to affine transformation of gray-values• Both small and large structures are detected
![Page 66: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/66.jpg)
Image Processing: Interest Points
Literature
66
There is a large body of literature on detectors and descriptors (later lecture)
A comparison paper (e.g. what is the most robust corner detectors):
K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir: A Comparison of Affine Region Detectors (IJCV 2006)
![Page 67: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/67.jpg)
Roadmap: Basics of Digital Image Processing
• Images
• Point operators (ch. 3.1)
• Filtering: (ch. 3.2, ch 3.3, ch. 3.4) – main focus
• Linear filtering
• Non-linear filtering
• Fourier Transformation (ch. 3.4)
• Multi-scale image representation (ch. 3.5)
• Edges detection and linking (ch. 4.2)
• Line detection (ch. 4.3)
• Interest Point detection (ch. 4.1.1)
• Using multiple Images: Define Challenges
06/11/2013Computer Vision I: Basics of Image Processing 67
![Page 68: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/68.jpg)
Difference between Appearance and Geometry
06/11/2013Computer Vision I: Basics of Image Processing 68
Does this look like a cotrrect match?
Image 1 Image 2Image 1 Image 2
Does this look like a cotrrect match?
Appearance based matching:
Geometry based matching: 1) Assume sensible camera model. We will see that: Given 4 matching
points on a surface defines how other points on the surface will match
Neighborhodd looks quite similiar (descriptor)
![Page 69: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/69.jpg)
The 3D case
Illustration of the general 3D case
06/11/2013Computer Vision I: Basics of Image Processing 69
Appearance based matching:
Geometry based matching: 1) Assume sensible camera model. We will see that: Given 7 matching
3D points defines how other 3D points match.
![Page 70: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/70.jpg)
Sparse versus Dense Matching: Tasks and Applications
06/11/2013Computer Vision I: Basics of Image Processing 70
Tasks:• Find places where we could match features
(points, lines, regions, etc)• Extract appearance - features descriptors• Find all possible (putative) appearance
matches between images• Verify with geometry
For what applications is sparse matching enough:• Sparse 3D reconstruction of a rigid scene• Panoramic stitching of a rotating / translating
camera
![Page 71: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/71.jpg)
Building Rome on a cloudless day
06/11/2013Computer Vision I: Basics of Image Processing 71
[Frahm et al. ECCV ´10 ]The old city of Dubrovnik
![Page 72: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/72.jpg)
Sparse versus Dense Matching
06/11/2013Computer Vision I: Basics of Image Processing 72
3D view interpolation
Kinect RGB and Depth data inputDense flow: frame 1->2
Dense flow: frame 2->1
Flow encoding
![Page 73: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/73.jpg)
Sparse versus Dense Matching: Tasks and Applications
06/11/2013Computer Vision I: Basics of Image Processing 73
Tasks (all in one):• Find for each pixel the 2D/3D/6D displacement
(using both appearance and geometry)• Find points which are occluded
For what applications is dense matching needed:• Dense reconstruction of rigid scene• 3D reconstruction of a non-rigid scene
[Goesele et al. ICCV 07]
![Page 74: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/74.jpg)
Using multiple Images: Define Challenges
A road map for the next five lectures
• L4: Geometry of a Single Cameraand Image Formation Process
• L5: Sparse Matching two images: Appearance
• L6: Sparse Matching two images: Geometry
• L7: Sparse Reconstructing the world (Geometry of n-views)
• L8: Dense Geometry estimation (stereo, flow and scene flow, registration)
06/11/2013Computer Vision I: Basics of Image Processing 74
![Page 75: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/75.jpg)
v
Outlook – matching 2 Images (appearance & geometry)
06/11/2013Computer Vision I: Basics of Image Processing 75
• Find interest points (including different scales)
• Find orientated patches around interest points to capture appearance
• Encode patch in a descriptor
• Find matching patches according to appearance (similar descriptors)
• Verify matching patches according to geometry
![Page 76: Computer Vision I - Algorithms and Applications: Basics of ...ds24/lehre/cv1_ws_2013/VL3.pdf · Computer Vision I - Algorithms and Applications: Basics of Digital Image Processing](https://reader033.fdocuments.in/reader033/viewer/2022042219/5ec5b3f3c8c5b435b46bfb8e/html5/thumbnails/76.jpg)
Reading for next class
This lecture:
• Chapter 3.5: multi-scale representation
• Chapter 4.2 and 4.3 - Edge and Line detection
• Chapter 4.1.1 Interest Point Detection
Next lecture:
• Chapter 2 (in particular: 2.1, 2.2) – Image formation process
• And a bit of Hartley and Zisserman – chapter 2
06/11/2013Computer Vision I: Basics of Image Processing 76