1Yossi CohenIntro to computer Vision

2

3

How to filter

2d Correlationh=filter2(g,f); or h=imfilter(f,g);

2d Convolutionh=conv2(g,f);

],[],[],[,

lnkmflkgnmhlk

f=imageg=filter

],[],[],[,

lnkmflkgnmhlk

4

Correlation filteringSay the averaging window size is 2k+1 x 2k+1:

Loop over all pixels in neighborhood around image pixel F[i,j]

Attribute uniform weight to each pixel

Now generalize to allow different weights depending on neighboring pixel’s relative position:

Non-uniform weights

5

Correlation filtering

Filtering an image: replace each pixel with a linear combination of its neighbors.

The filter “kernel” or “mask” H[u,v] is the prescription for the weights in the linear combination.

This is called cross-correlation, denoted

6

Properties of smoothing filters

Smoothing Values positive Sum to 1 constant regions same as input Amount of smoothing proportional to mask size Remove “high-frequency” components; “low-pass” filter

7

Filtering an impulse signal

0 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0

a b c

d e fg h i

What is the result of filtering the impulse signal (image) F with the arbitrary kernel H?

?

8

Filtering an impulse signal

0 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 1 0 0 00 0 0 0 0 0 00 0 0 0 0 0 00 0 0 0 0 0 0

a b c

d e fg h i

0 0 0 0 0 0 00 0 0 0 0 0 00 0 i h g 0 00 0 f e d 0 00 0 c b a 0 00 0 0 0 0 0 00 0 0 0 0 0 0

What is the result of filtering the impulse signal (image) F with the arbitrary kernel H?

9

Averaging filter What values belong in the kernel H for the moving

average example?

0 10 20 30 30

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

111

111

111

“box filter”

?

10

Smoothing by averagingdepicts box filter: white = high value, black = low value

original filtered

11

Gaussian filter

0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 0 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 0 0 0 0 0 0 00 0 90 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0

1 2 1

2 4 21 2 1

This kernel is an approximation of a 2d Gaussian function:

12

Smoothing with a Gaussian

13

Smoothing with a Box

14

Weight contributions of neighboring pixels by nearness

0.003 0.013 0.022 0.013 0.0030.013 0.059 0.097 0.059 0.0130.022 0.097 0.159 0.097 0.0220.013 0.059 0.097 0.059 0.0130.003 0.013 0.022 0.013 0.003

5 x 5, = 1

Slide credit: Christopher Rasmussen

Gaussian

15

Gaussian filtersWhat parameters matter here?Size of kernel or mask

Note, Gaussian function has infinite support, but discrete filters use finite kernels

σ = 5 with 10 x 10 kernel

σ = 5 with 30 x 30 kernel

16

Gaussian filtersWhat parameters matter here?Variance of Gaussian: determines extent of

smoothing

σ = 2 with 30 x 30 kernel

σ = 5 with 30 x 30 kernel

17

Matlab>> hsize = 10;>> sigma = 5;>> h = fspecial(‘gaussian’ hsize, sigma);

>> mesh(h);

>> imagesc(h);

>> outim = imfilter(im, h); % correlation >> imshow(outim);

outim

18

Smoothing with a Gaussian

for sigma=1:3:10 h = fspecial('gaussian‘, fsize, sigma);out = imfilter(im, h); imshow(out);pause;

end

…

Parameter σ is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing.

19

Gaussian filters

• Remove “high-frequency” components from the image (low-pass filter)Images become more smooth

• Convolution with self is another Gaussian–So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have

–Convolving two times with Gaussian kernel of width σ is same as convolving once with kernel of width σ√2

• Separable kernel–Factors into product of two 1D Gaussians

Source: K. Grauman

20

Separability of the Gaussian filter

Source: D. Lowe

21

Separability example

*

*

=

=

2D convolution(center location only)

Source: K. Grauman

The filter factorsinto a product of 1D

filters:

Perform convolutionalong rows:

Followed by convolutionalong the remaining column:

22

Convolution Convolution:

Flip the filter in both dimensions (bottom to top, right to left) Then apply cross-correlation

Notation for convolution operator

F

H

23

Convolution vs. correlationConvolution

Cross-correlation

24

Key properties of linear filters

Linearity: filter(f1 + f2) = filter(f1) + filter(f2)

Shift invariance: same behavior regardless of pixel location

filter(shift(f)) = shift(filter(f))

Any linear, shift-invariant operator can be represented as a convolution

Source: S. Lazebnik

25

More properties• Commutative: a * b = b * a

Conceptually no difference between filter and signalBut particular filtering implementations might break this equality

• Associative: a * (b * c) = (a * b) * cOften apply several filters one after another: (((a * b1) * b2) * b3)This is equivalent to applying one filter: a * (b1 * b2 * b3)

• Distributes over addition: a * (b + c) = (a * b) + (a * c)

• Scalars factor out: ka * b = a * kb = k (a * b)

• Identity: unit impulse e = [0, 0, 1, 0, 0],a * e = a Source: S. Lazebnik

26

MatlabLab 1 – Smooth/Blur Filters

27

Lets try to blur filter%%basic image filtersimrgb = imread('peppers.png');imshow(imrgb);a = [1 1 1; 1 1 1; 1 1 1]Corroutimg = filter2(a, imgray);Convoutimg = conv2(imgray,a);figure;imshow(Corroutimg)figureimshow(Convoutimg)

28

Fix itCan we conv2 an RGB image??Use rgb2grayWhats wrong now?Try preserving the image powerConvert image to double

29

Results%%basic image filtersimrgb = imread('peppers.png');imgray = im2double(rgb2gray(imrgb));imshow(imgray);a = 1/16*ones(4) Corroutimg = filter2(a, imgray);Convoutimg = conv2(a, imgray);figure;imshow(Corroutimg)figureimshow(Convoutimg)

30

Sharp%%basic sharp filterimrgb = imread('peppers.png');imgray = im2double(rgb2gray(imrgb));imshow(imgray);a = [ 0 0 0; 0 2 0; 0 0 0] - 1/9*ones(3)Corroutimg = filter2(a, imgray);figure;imshow(Corroutimg)

31

Practical mattersWhat happens near the edge?

the filter window falls off the edge of the imageneed to extrapolatemethods:

clip filter (black)wrap aroundcopy edgereflect across edge

Source: S. Marschner

32

Matlab edge aware filtering

methods (MATLAB):clip filter (black): imfilter(f, g, 0)wrap around: imfilter(f, g, ‘circular’)copy edge: imfilter(f, g, ‘replicate’) reflect across edge: imfilter(f, g, ‘symmetric’)

33

Practical matters

What is the size of the output?• MATLAB: filter2(g, f, shape)

shape = ‘full’: output size is sum of sizes of f and gshape = ‘same’: output size is same as fshape = ‘valid’: output size is difference of sizes of f and g

f

gg

gg

f

gg

gg

f

gg

gg

full same valid

34

Median filtersA Median Filter operates over a window by

selecting the median intensity in the window.What advantage does a median filter have

over a mean filter?Is a median filter a kind of convolution?

35

MatlabLab 2 – Median Filter

36

Comparison: salt and pepper noise

37

Median Filter Example%%Medianimrgb = imread('peppers.png');imgray = im2double(rgb2gray(imrgb));imnoise = imnoise(imgray,'salt & pepper',0.1);imshow(imnoise)figureimshow(medfilt2(imnoise))

38

Basic Filters summary

Linear filtering is sum of dot product at each positionCan smooth, sharpen, translate

(among many other uses)

Be aware of details for filter size, extrapolation, cropping

111

111

111