Linear filtering
description
Transcript of Linear filtering
![Page 1: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/1.jpg)
Linear filtering
![Page 2: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/2.jpg)
Motivation: Image denoising• How can we reduce noise in a photograph?
![Page 3: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/3.jpg)
• Let’s replace each pixel with a weighted average of its neighborhood
• The weights are called the filter kernel• What are the weights for the average of a
3x3 neighborhood?
Moving average
111
111
111
“box filter”
Source: D. Lowe
![Page 4: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/4.jpg)
Defining convolution
lk
lkglnkmfnmgf,
],[],[],)[(
f
• Let f be the image and g be the kernel. The output of convolving f with g is denoted f * g.
Source: F. Durand
• MATLAB functions: conv2, filter2, imfilter
Convention: kernel is “flipped”
![Page 5: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/5.jpg)
Key properties• Linearity: filter(f1 + f2) = filter(f1) + filter(f2)• Shift invariance: same behavior regardless of
pixel location: filter(shift(f)) = shift(filter(f))• Theoretical result: any linear shift-invariant
operator can be represented as a convolution
![Page 6: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/6.jpg)
Properties in more detail• Commutative: a * b = b * a
• Conceptually no difference between filter and signal
• Associative: a * (b * c) = (a * b) * c• Often apply several filters one after another: (((a * b1) * b2) * b3)• This is equivalent to applying one filter: a * (b1 * b2 * b3)
• Distributes over addition: a * (b + c) = (a * b) + (a * c)• Scalars factor out: ka * b = a * kb = k (a * b)• Identity: unit impulse e = […, 0, 0, 1, 0, 0, …],
a * e = a
![Page 7: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/7.jpg)
Annoying detailsWhat is the size of the output?• MATLAB: filter2(g, f, shape)
• shape = ‘full’: output size is sum of sizes of f and g• shape = ‘same’: output size is same as f• shape = ‘valid’: output size is difference of sizes of f and g
f
gg
gg
f
gg
gg
f
gg
gg
full same valid
![Page 8: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/8.jpg)
Annoying detailsWhat about near the edge?
• the filter window falls off the edge of the image• need to extrapolate• methods:
– clip filter (black)– wrap around– copy edge– reflect across edge
Source: S. Marschner
![Page 9: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/9.jpg)
Annoying detailsWhat about near the edge?
• the filter window falls off the edge of the image• need to extrapolate• methods (MATLAB):
– clip filter (black): imfilter(f, g, 0)– wrap around: imfilter(f, g, ‘circular’)– copy edge: imfilter(f, g, ‘replicate’)– reflect across edge: imfilter(f, g, ‘symmetric’)
Source: S. Marschner
![Page 10: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/10.jpg)
Practice with linear filters
000010000
Original
?
Source: D. Lowe
![Page 11: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/11.jpg)
Practice with linear filters
000010000
Original Filtered (no change)
Source: D. Lowe
![Page 12: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/12.jpg)
Practice with linear filters
000100000
Original
?
Source: D. Lowe
![Page 13: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/13.jpg)
Practice with linear filters
000100000
Original Shifted leftBy 1 pixel
Source: D. Lowe
![Page 14: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/14.jpg)
Practice with linear filters
Original
?111111111
Source: D. Lowe
![Page 15: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/15.jpg)
Practice with linear filters
Original
111111111
Blur (with abox filter)
Source: D. Lowe
![Page 16: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/16.jpg)
Practice with linear filters
Original
111111111
000020000 - ?
(Note that filter sums to 1)
Source: D. Lowe
![Page 17: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/17.jpg)
Practice with linear filters
Original
111111111
000020000 -
Sharpening filter- Accentuates differences
with local average
Source: D. Lowe
![Page 18: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/18.jpg)
Sharpening
Source: D. Lowe
![Page 19: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/19.jpg)
SharpeningWhat does blurring take away?
original smoothed (5x5)
–
detail
=
sharpened
=
Let’s add it back:
original detail
+
![Page 20: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/20.jpg)
Smoothing with box filter revisited• What’s wrong with this picture?• What’s the solution?
Source: D. Forsyth
![Page 21: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/21.jpg)
Smoothing with box filter revisited• What’s wrong with this picture?• What’s the solution?
• To eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center
“fuzzy blob”
![Page 22: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/22.jpg)
Gaussian Kernel
• Constant factor at front makes volume sum to 1 (can be ignored when computing the filter values, as we should renormalize weights to sum to 1 in any case)
0.003 0.013 0.022 0.013 0.0030.013 0.059 0.097 0.059 0.0130.022 0.097 0.159 0.097 0.0220.013 0.059 0.097 0.059 0.0130.003 0.013 0.022 0.013 0.003
5 x 5, = 1
Source: C. Rasmussen
![Page 23: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/23.jpg)
Gaussian Kernel
• Standard deviation : determines extent of smoothing
Source: K. Grauman
σ = 2 with 30 x 30 kernel
σ = 5 with 30 x 30 kernel
![Page 24: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/24.jpg)
Choosing kernel width• The Gaussian function has infinite support,
but discrete filters use finite kernels
Source: K. Grauman
![Page 25: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/25.jpg)
Choosing kernel width• Rule of thumb: set filter half-width to about 3σ
![Page 26: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/26.jpg)
Gaussian vs. box filtering
![Page 27: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/27.jpg)
Gaussian filters• Remove high-frequency components from the
image (low-pass filter)• Convolution with self is another Gaussian
• So can smooth with small- kernel, repeat, and get same result as larger- kernel would have
• Convolving two times with Gaussian kernel with std. dev. σ is same as convolving once with kernel with std. dev.
• Separable kernel• Factors into product of two 1D Gaussians• Discrete example:
Source: K. Grauman
2
121121
121242121
![Page 28: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/28.jpg)
Separability of the Gaussian filter
Source: D. Lowe
![Page 29: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/29.jpg)
Why is separability useful?• Separability means that a 2D convolution can
be reduced to two 1D convolutions (one among rows and one among columns)
• What is the complexity of filtering an n×n image with an m×m kernel? • O(n2 m2)
• What if the kernel is separable?• O(n2 m)
![Page 30: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/30.jpg)
Noise
• Salt and pepper noise: contains random occurrences of black and white pixels
• Impulse noise: contains random occurrences of white pixels
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Source: S. Seitz
![Page 31: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/31.jpg)
Gaussian noise• Mathematical model: sum of many
independent factors• Good for small standard deviations• Assumption: independent, zero-mean noise
Source: M. Hebert
![Page 32: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/32.jpg)
Smoothing with larger standard deviations suppresses noise, but also blurs the image
Reducing Gaussian noise
![Page 33: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/33.jpg)
Reducing salt-and-pepper noise
What’s wrong with the results?
3x3 5x5 7x7
![Page 34: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/34.jpg)
Alternative idea: Median filtering
• A median filter operates over a window by selecting the median intensity in the window
• Is median filtering linear?Source: K. Grauman
![Page 35: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/35.jpg)
Median filter• What advantage does median filtering have
over Gaussian filtering?• Robustness to outliers
Source: K. Grauman
![Page 36: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/36.jpg)
Median filterSalt-and-pepper noise Median filtered
Source: M. Hebert
MATLAB: medfilt2(image, [h w])
![Page 37: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/37.jpg)
Gaussian vs. median filtering3x3 5x5 7x7
Gaussian
Median
![Page 38: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/38.jpg)
Sharpening revisited
Source: D. Lowe
![Page 39: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/39.jpg)
Sharpening revisitedWhat does blurring take away?
original smoothed (5x5)
–
detail
=
sharpened
=
Let’s add it back:
original detail
+ α
![Page 40: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/40.jpg)
Unsharp mask filter
Gaussianunit impulse
Laplacian of Gaussian
))1(()1()( gefgffgfff
image blurredimage
unit impulse(identity)
![Page 41: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/41.jpg)
Application: Hybrid Images
A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
![Page 42: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/42.jpg)
![Page 43: Linear filtering](https://reader030.fdocuments.in/reader030/viewer/2022033021/56816388550346895dd47807/html5/thumbnails/43.jpg)
Application: Hybrid Images
A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Gaussian Filter
Laplacian Filter