Spatial filtering

Stefano Ferrari

Universita degli Studi di Milanostefano.ferrari@unimi.it

Methods for Image Processing

academic year 2016–2017

Filtering

I The term filtering refers to a technique developed for thefrequency domain.

I Rapid variations of intensity can be associated to highfrequency components, while slow changes can be associatedto low frequency components.

I The effects of the filtering process is the attenuation or theenhancement of the components in a target interval offrequency.

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 1

Spatial filtering

I Similar effects can be obtained also in the spatial domain:techniques of this kind are defined as spatial filtering.

I In general, the spatial filtering techniques operate on an imagetaking into consideration the intensity values in an suitableneighborhood of each pixel.

I For each pixel of the original image, the intensity of thecorresponding pixel in the “filtered” image is computed.

I The transformation rule is often described by a matrix, calledfilter, with the same size of the considered neighborhood.

I If the transforming rule is a linear function of the intensities inthe considered neighborhood, the technique is called spatiallinear filtering (non-linear, otherwise).

Linear filtering

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 2

Linear filtering (2)

I The pixel (x , y) in the filtered image, g , is obtained as theweighted average of the pixels in the original image, f , of asuitable neighborhood of (x , y):

g(x , y) =a∑

s=−a

b∑

t=−b

w(s, t) f (x + s, y + t)

I The weight matrix, w , is called filter.I Or mask, template, window.I For convenience, often a matrix with an odd number of rows

2a + 1, and columns, 2b + 1, is used.

Correlation and convolution

I Correlation

g(x , y) = w(x , y)?f (x , y) =a∑

s=−a

b∑

t=−b

w(s, t) f (x +s, y +t)

I Convolution

g(x , y) = w(x , y)∗f (x , y) =a∑

s=−a

b∑

t=−b

w(s, t) f (x−s, y−t)

I Vectorial representation

R = w1z1 + · · ·+ wmnzmn =mn∑

k=1

wkzk = wTz

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 3

Correlation

f w0 0 0 1 0 0 0 0 1 2 3 2 8

↓f 0 0 0 1 0 0 0 0w 1 2 3 2 8

↑ starting position alignment

zero paddingf 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8

Correlation (2)

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 4

Correlation (3)

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8 2

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8 2 3

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8 2 3 2

Correlation (4)

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8 2 3 2 1

. . .

f 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0w 1 2 3 2 8w ? f 0 0 0 8 2 3 2 1 0 0 0 0

w ? f 0 0 0 8 2 3 2 1 0 0 0 0Cropping 0 8 2 3 2 1 0 0f 0 0 0 1 0 0 0 0

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 5

Convolution

f w rotated 180◦

0 0 0 1 0 0 0 0 8 2 3 2 1

↓f 0 0 0 1 0 0 0 0w 8 2 3 2 1

↑ starting position alignment

. . .

w ∗ f 0 0 0 1 2 3 2 8 0 0 0 0Cropping 0 1 2 3 2 8 0 0f 0 0 0 1 0 0 0 0

Correlation and convolution on images (1)

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 6

Correlation and convolution on images (2)

Vectorial representation

I When relative position of the coefficients is not important, itis more convenient to represent the effect of the filter using avectorial representation.

I A conventional indexing have to be specified.

R = w1z1 + · · ·+ wmnzmn =mn∑

k=1

wkzk = wTz

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 7

Filter specification

I The filter coefficients have to be specified.I Their values depend on the desired effect.

I Direct specification:

R =1

9

9∑

i=1

zi ⇒ wi =1

9, ∀i

I Function based specification:

h(x , y) = e−x2+y2

2σ2 ⇒ w(s, t) = h(s, t)

I Algorithmic specificationI Non linear filtersI e.g., “max” filter

Smoothing filters

I Results in defocused images (blur).I Small details (smaller that the size of the filter) removal:

I more attention on large objects;I small gaps bridging;I noise reduction.

I Depending on the type of noise that affects the image, thefilter can be linear or non-linear.

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 8

Linear smoothing filters

I Averaging filters (or low pass)

I Arithmetic average

I Weighted average

Smoothing effects (1)

Original image and image filtered using a 3× 3 averaging filter.

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 9

Smoothing effects (2)

Filtering with averaging filters (5× 5 and 9× 9).

Smoothing effects (3)

Filtering with averaging filters (15× 15 and 35× 35).

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 10

Example: details removal

f g = f ∗ h m = g > θ

k

a b c

d(a) f , image 353× 382

(b) g , image filtered with13× 13 smoothing filter

(c) m, thresholded image

(d) k , masked image:k(x , y) = f (x , y)×m(x , y)

Non-linear filters

I Statistics-based filters are the most used non-linear filters.

I Ranking (which requires sorting) is more computationallyexpensive than linear operations.

I In particular:I Median filterI Max and Min filtersI Percentiles based filters

I Example: f = [100, 120, 98, 99, 110, 255, 100, 200, 10]I mean: 121I median: 100

I sorted: [10, 98, 99, 100, 100, 110, 120, 200, 255]

I min: 10I max: 255

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 11

Example: median filter

Salt-and-pepper noise removal

a b c

(a) Original image, corrupted by salt-and-pepper noise.

(b) Image filtered with averaging filter 3× 3.

(c) Image filtered with median filter 3× 3.

Homeworks and suggested readings

DIP, Sections 3.4, 3.5

I pp. 144–156

GIMPI Filters

I BlurI Enhance

I Despeckle

I GenericI Convolution matrix

.

Stefano Ferrari— Methods for Image processing— a.a. 2016/17 12