Post on 24-Apr-2018
Table of Content
Digital Image ProcessingImage Filtering
O. Le Meurolemeur@irisa.fr
Univ. of Rennes 1http://www.irisa.fr/temics/staff/lemeur/
January 2, 2011
1
Table of Content
1 Introduction
2 Point-to-point transformation
3 Linear �ltering (neighborhoodoperator)
4 Non Linear �ltering
5 Conclusion
2
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Introduction
1 Introduction
2 Point-to-point transformation
3 Linear �ltering (neighborhoodoperator)
4 Non Linear �ltering
5 Conclusion
3
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Introduction: image transformation
There exist 3 types of transformation:
Point to point transformation:The output value at a speci�c coordinateis dependent only on one input value butnot necessarily at the same coordinate;
Local to point transformation:The output value at a speci�c coordinateis dependent on the input values in theneighborhood of that same coordinate;
Global to point transformation:The output value at a speci�c coordinateis dependent on all the values in the inputimage.
Note that the complexity increases with the size of the considered neighborhood...
4
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Point-to-point transformation
1 Introduction
2 Point-to-point transformationSpatial coordinates-basedtransformationsPixel values-based transformations
3 Linear �ltering (neighborhoodoperator)
4 Non Linear �ltering
5 Conclusion
5
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Spatial coordinates-based transformations
Remark:
This section is composed of several pictures extracted fromhttp://eeweb.poly.edu/~onur/lectures/lectures.html.
Let im[x, y ] be an input image of size N × N.
A spatial coordinates-based transformation, also called warping, aims at providing animage IM[k, l ] from the input image im[x, y ]:
IM[k, l ] = im[x(k, l), y(k, l)]
x(k, l) and y(k, l) are the transformations or the pixel warping functions. These functionsjust modify the spatial coordinates of a pixel not its value;
Special cases to take into consideration:
the new coordinates [x(k, l), y(k, l)] is out of the image IM. In this case,IM[k, l ] = 0;the new coordinates must be integers (Rounding operation, nearest integers...).
6
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Transpose
The transpose tansformation is given by
x(k, l) = l
y(k, l) = k
7
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Vertical Flip
The vertical �ip tansformation is given by
x(k, l) = N − k
y(k, l) = l
The horizontal �ip tansformation is given by
x(k, l) = k
y(k, l) = N − l
8
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Translation
The translation tansformation is given by
x(k, l) = k + Tk
y(k, l) = l + Tl
where Tk and Tl are the transalation values for the x-axis and the y-axis respectively.
In the example below, we have (Tl = −50)
IM[k, l ] = im[x(k, l), y(k, l)]
IM[k, l ] = im[k, l + Tl ]
Di�erent transformations can be obtained depending on Tk and Tl values.
9
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Rotation
The rotation tansformation is given by
x(k, l) = (k − x0)cosθ − (l − y0)sinθ + x0
y(k, l) = (k − x0)sinθ + (l − y0)cosθ + y0
where [x0, y0] is the spatial coordinates of the center of the rotation and θ the angle.
Extracted from http://eeweb.poly.edu/ onur/lectures/lectures.html.
10
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Wave
x(k, l) = k
y(k, l) = l + α× sin(β × l)
x(k, l) = k + α× sin(β × k)
y(k, l) = l
α and β can be used to strengthen the e�ect.
11
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Warp and swirl
x(k, l) = k
y(k, l) =sign(l − y0)
y0× (l − y0)2 + y0
The swirl e�ect is a rotation but theangle of the rotation θ varies with thepixel distance from the center of theimage [x0, y0]:
d =√
(k − x0)2 + (l − y0)2
θ =π
512r
If r −→ 0, θ is small...
12
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Glass e�ect
A glass e�ect is obtained by adding a small and random displacement to each pixel:
x(k, l) = k + (RAND(1, 1)− 1/2)× 10
y(k, l) = l + (RAND(1, 1)− 1/2)× 10
Extracted from http://eeweb.poly.edu/ onur/lectures/lectures.html.
13
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Summary
All 2D linear transformations: [x
y
]=
[a b
c d
] [k
l
]Scale, rotation, mirror...
Properties:
Origin maps to origin;
Lines map to lines;
Ratios are preserved...
A�ne transformations (linear transf. + translation):xyw
=
a b c
d e f
0 0 1
klw
Properties:
Origin does not necessarily map to origin;
Lines map to lines;
Ratios are preserved...
14
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Summary
A�ne transformations (linear transf. + translation):
Translation: xy1
=
1 0 tx0 1 ty0 0 1
kl1
Scale: xy
1
=
sx 0 00 sy 00 0 1
kl1
2D in-plane rotation: xy
1
=
cosθ −sinθ 0sinθ cosθ 00 0 1
kl1
15
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Pixel values-based transformations
Let im[x , y ] be an input image of size N × N.
A pixel values-based transformation aims at providing an image IM[k, l ] from theinput image im[x , y ]:
IM[k, l ] = f (im[k, l ])
Noticed that the spatial coordinates of pixel are not modi�ed. The function f isused to modify the pixel values.
The simplest one is the identity function: f (p) = p:
IM[k, l ] = f (im[k, l ])
IM[k, l ] = im[k, l ]
16
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Histogram
An image histogram is a graphical representation of the tonal distribution in a digitalimage. It plots the number of pixels for each tonal value.
Histogram gives information about the global distribution of an image.
Histogram plots the number of pixels in the image (vertical axis) with a particularbrightness value (horizontal axis).
(a) Original (b) Histogram (intensity)
17
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Histogram
(a) Original (b) Histogram (intensity)
(c) Original (d) Histogram (intensity)
18
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Histogram
High dynamic range in the last caseproviding the best quality.
From R.C. Gonzalez, R.E. Woods, Digital image processing.
19
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Histogram equalization
The goal is to increase the global contrast of images, especially when the usable dataof the image is represented by close contrast values.
Consider a discrete grayscale image {x} and let ni be the number of occurrences ofgray level i .
The probability of an occurrence of a pixel of level i in the image ispx (i) = p(x = i) =
nin, 0 ≤ i < L, L being the total number of gray levels in the
image, n being the total number of pixels in the image.
Let us also de�ne the cumulative distribution function: cdfx (i) =∑i
j=0 px (j).
We want to produce an image {y}, such that cdfy (i) = iK .
y = cdfx (x)× (max {x} −min {x}) + min {x}
20
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Negative image
The negative image is obtained by f (p) = 255− p (pixel values are coded on 8 bits).
IM[k, l ] = f (im[k, l ])
IM[k, l ] = 255− im[k, l ]
21
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Piece continuous transformation
The objective of such transformation is:
to compress pre-determined ranges of values: Range compression;
to accentuate pre-determined ranges of values: Range stretching.
f (p) =
α1 × p 0 ≤ p < a1α2 × (p − a1) a1 ≤ p < a2...
αi (p − ai−1) +(∑i−1
j=1 αj (αj − αj−1)
ai−1 ≤ p < ai
...
Obviously, we have:
αi < 1, compression;
αi > 1, stretching.
22
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Piece continuous transformation
Example of contrast stretching:
Let f a function de�ned on three pieces:
f (p) =
α1p, 0 ≤ p < a1α2(p − a1) + α1a1, a1 ≤ p < a2α3(p − a2) + (α2(a2 − a1) + α1a1), a2 ≤ p ≤ 255
23
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Piece continuous transformation
Example of contrast stretching:
Let imaging that α1 and α2 are null. The �ltered image contains only grey levelbelonging to [a1, a2]. We just keep a slice of the image.
24
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Spatial coordinates-based transformationsPixel values-based transformations
Piece continuous transformation
Binary thresholding:
IM[x , y ] =
{0 im[x,y]<T255 otherwise
Gamma correction:
IM[x , y ] = im[x , y ]γ
25
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Linear �ltering with neighborhoods operator
1 Introduction
2 Point-to-point transformation
3 Linear �ltering (neighborhoodoperator)De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatialdomainFrequency domain �ltering
4 Non Linear �ltering
5 Conclusion
26
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
De�nition
De�nition of neighborhood around a pixel of spatial coordinate (x , y). Theneighborhood is called V (x , y):
Two examples:
27
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
De�nition
2D Finite Impluse Response (FIR) �lter:
IM[x , y ] = (im ∗ h)[x , y ]
IM[x , y ]︸ ︷︷ ︸Output
=∑
k∈V (x,y)
∑l∈V (x,y)
h(k, l)︸ ︷︷ ︸Filter coe�.
im[x − k, y − l ]︸ ︷︷ ︸Input
with h is the 2D impulse response called also the Point Spread Function (PSF) or thekernel of the transform. It is composed of the �lter coe�cients (�nite length).
The gain of the �lter is equal to
g =∑
k∈V (x,y)
∑l∈V (x,y)
h(i , j)
28
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
De�nition
IM[x , y ]︸ ︷︷ ︸Output
=∑
k∈V (x,y)
∑l∈V (x,y)
h(k, l)︸ ︷︷ ︸Filter coe�.
im[x − k, y − l ]︸ ︷︷ ︸Input
where H is the convolution kernel.
Example: the �lter support is (3× 5). The convolution kernel is:
H =
h(−2,−1) h(−1,−1) h(0,−1) h(1,−1) h(2,−1)h(−2, 0) h(−1, 0) h(0, 0) h(1, 0) h(2, 0)h(−2,−1) h(−1,−1) h(0,−1) h(1,−1) h(2,−1)
29
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
De�nition
Example for a neigborhood of size (2N + 1)× (2N + 1):
IM[x , y ] =N∑
k=−N
N∑l=−N
h(k, l)im[x−k, y−l ]
Number of multiplications per output point: O(N2)
N=2
30
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Average �lter
The most simple low-pass �lter is the local averaging operation. The main e�ect of alow-pass �lter is a blurring. The size of the kernel is (2N + 1)× (2N + 1):
h(k, l) =
{1
(2N+1)2−N ≤ k, l ≤ −N
0 Otherwise
For N = 1, the convolution kernel is given by:
H =1
9
1 1 11 1 11 1 1
31
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Average �lter
Three examples of averaging for di�erent sizes of kernel. From the left-hand side tothe right-side, N = {1, 3, 8}:
The amount of blur increases with the size of the kernel (the number of operation tooO(N2)). In order to �lter pixels located near the edges of the image, edge pixel valuesare replicated to give su�cient data (this is not the case here).
32
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian �lter
The kernel h is given by the following function
h(x , y) =1
2πσ2exp
(−x2 + y2
2σ2
)
Each pixel's new value is set to a weighted average of that pixel's neighborhood. Theoriginal pixel's value receives the heaviest weight (having the highest Gaussian value)and neighboring pixels receive smaller weights as their distance to the original pixelincreases. This results in a blur that preserves boundaries and edges better than other,more uniform blurring �lters.Note that the �lter support is truncated...
33
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian �lter
h(x , y) =1
2πσ2exp
(−x2 + y2
2σ2
)Example of kernel:
σ = 0.84089642, N = 3:Note that the center element (at [4, 4]) has the largest value, decreasingsymmetrically as distance from the center increases.
H =
0.00000067 0.00002292 0.00019117 0.00038771 0.00019117 0.00002292 0.000000670.00002292 0.00078633 0.00655965 0.01330373 0.00655965 0.00078633 0.000022920.00019117 0.00655965 0.05472157 0.11098164 0.05472157 0.00655965 0.000191170.00038771 0.01330373 0.11098164 0.22508352 0.11098164 0.01330373 0.000387710.00019117 0.00655965 0.05472157 0.11098164 0.05472157 0.00655965 0.000191170.00002292 0.00078633 0.00655965 0.01330373 0.00655965 0.00078633 0.000022920.00000067 0.00002292 0.00019117 0.00038771 0.00019117 0.00002292 0.00000067
Note that 0.22508352 (the central one) is 1177 times larger than 0.00019117which is just outside 3σ.
34
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian �lter
h(x , y) =1
2πσ2exp
(−x2 + y2
2σ2
)Example of kernel:
σ = 0.6, N = 1:
H =1
16
1 2 12 4 21 2 1
σ = 1, N = 2:
H =1
1444
1 9 18 9 19 81 162 81 918 162 324 162 189 81 162 81 91 9 18 9 1
35
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian �lter
Example of Gaussian �ltering with σ = 2:
36
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian �lter
Increasing sigma increases the smoothing
37
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Other low pass �lters
2D Pyramidal �lter:
H =1
81
1 2 3 2 12 4 6 4 23 6 9 6 32 4 6 4 21 2 3 2 1
Conic �lter:
H =1
25
0 0 1 0 00 2 2 2 01 2 5 2 10 2 2 2 00 0 1 0 0
38
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
High-pass �lter
The high-pass �ltered image can be thought of as the original image minus the lowpass �ltered image.
IM[x , y ] = im[x , y ]−N∑
k=−N
N∑l=−N
h(k, l)im[x − k, y − l ]
IM[x , y ] =N∑
k=−N
N∑l=−N
h(k, l)im[x − k, y − l ]
with h the convolution kernel.
H =
0 −1 0−1 5 −10 −1 0
H =
−1 −1 −1−1 9 −1−1 −1 −1
H =
1 −2 1−2 5 −21 −2 1
39
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
High-pass �lter
Three examples of high-pass �ltering for di�erent sizes of kernel. From the left-handside to the right-side, N = {1, 3, 8}:
When the kernel's size increases, the �ltering is more important and then the result isless noisy.
40
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
f ′(x) = lim∆x→0
f (x + ∆x)− f (x)
∆x
Local variations of intensity are an important source of information in imageprocessing. These local variations are gradient (it measures the rate of change of thefunction):
∇im[k, l ] =
[∂im
∂x[k, l ],
∂im
∂y[k, l ]
]In the illustration below: GX = ∂im
∂x[k, l ] and GY = ∂im
∂y[k, l ].
41
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
GX = ∂im∂x
[k, l ]: Kernel = hx =[−1 1
]∂im
∂x[k, l ] ≈ im[k + 1, l ]− im[k, l ]
GY = ∂im∂y
[k, l ] Kernel = hy =[−1 1
]T∂im
∂y[k, l ] ≈ im[k, l + 1]− im[k, l ]
However, most of the time, we use the following kernel[−1 0 1
]and[
−1 0 1]T
(phase =zero). Below, from the left-hand side to the right: original,[−1 1
],[−1 0 1
].
42
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
However, these �lters are very sensitive to the noise. In order to enhance therobustness, these �lters are combined with a blurring �lter:
hx =
−1 0 1−2 0 2−1 0 1
hy =
−1 −2 −10 0 01 2 1
These kernel are the Sobel's kernel. The blurring �lter is
[1 2 1
].
IMx [k, l ] = (im ∗ hx )[k, l ]
IMy [k, l ] = (im ∗ hy )[k, l ]
In the same vein, Prewitt and Robert's �lters.
43
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
Norm of the gradient: ‖∇IM[k, l ]‖2 =√IMx [k, l ]2 + IMy [k, l ]2;
Its orientation: arg(∇IM[k, l ]) = arctan(IMy [k,l ]
IMx [k,l ]
)From left-hand side to the right: IMx , IMy and the norm.
44
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
The Laplacian of a picture is the second derivative:
∂2im
∂2x[k, l ] ≈ im[k + 1, l ] + im[k − 1, l ]− 2× im[k, l ]
∂2im
∂2y[k, l ] ≈ im[k, l + 1] + im[k, l − 1]− 2× im[k, l ]
∇2im[k, l ] =∂2im
∂2x[k, l ] +
∂2im
∂2y[k, l ]
∇2im[k, l ] ≈ im[k + 1, l ] + im[k − 1, l ] + im[k, l + 1] + im[k, l − 1]− 4× im[k, l ]
For a 4-neighborhood, the kernel is given by
h =
0 1 01 −4 10 1 0
We can extent this kernel to compute the laplacian in all directions (8-neighborhood):
h =
1 1 11 −8 11 1 1
45
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Di�erentiation �lter
The second order derivatives have a stronger response to �ne details (e.g. thin lines)than the �rst order derivatives.
46
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Frequency domain �ltering
where H is the convolution kernel.
IM[x , y ] = (im ∗ h)[x , y ]
im1[x , y ] ∗ im2[x , y ]F→ IM1[u, v ]× IM2[u, v ]
im1[x , y ]× im2[x , y ]F→ IM1[u, v ] ∗ IM2[u, v ]
When the size of the kernel is large, it is better to apply the �lter in the frequency domain.
For more information:Digital Image Processing, by R. C. Gonzalez and R. E. Woods, 3rd edition, Pearson PrenticeHall, 2008.
47
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Frequency domain �ltering
We can spatially �lter an image by Fourier transforming and applying a frequency�lter:
IM[x , y ] = im[x , y ] ∗ h[x , y ]
˜IM[u, v ] = IM[u, v ]× H[u, v ]
where, H[u, v ] is the �lter in the frequency function.
48
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Ideal low pass �lter
From the left-hand side to the right: Ideal low pass �lter transfert function, �lterdisplayed as an image, �lter radial cross section.
H(u, v) =
{1 D(u, v) ≤ D0
0 D(u, v) > D0
With D the euclidean distance from the spectrum center (N2, N2
).
49
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Ideal low pass �lter
Low pass �ltering:
Ringing and blurring
50
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Butterworth low pass �lter
H(u, v) =1
1 +(D(u,v)D0
)2n
51
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Butterworth low pass �lter
Top: spatial representation of the �lter for di�erent orders;Bottom: intensity pro�les through the center of the �lters.
Butterworth low pass �ltering:
Smooth transition in blurring, no ringing is present.
52
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Gaussian low pass �lter
H(u, v) = exp
(−D2(u, v)
2D20
)with D0 = σ. Gaussian low pass �ltering:
Smooth transition in blurring, no ringing is present.
53
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Low pass �lter
Comparison between the ideal, the Butterworth and the gaussian low pass �lter:
Ideal
Butterworth
Gaussian
54
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
High pass �ltering in the frequency domain
HHP(u, v) = 1− HLP(u, v)
55
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
High pass �ltering in the frequency domain
Ideal high-pass �lters enhance edges but su�er from ringing artefacts, just likeIdeal LPF;
smoother results with the two others.
56
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Frequency domain �ltering
Example of two band-pass �lters:
57
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Laplacian in the frequency domain
We remind (see previous lecture):
∇2im[k, l ] =∂2im
∂2x[k, l ] +
∂2im
∂2y[k, l ]
dnx(t)dtn
F−→ (j2πf )nX (f ).
From this, it follows that
∂2im
∂2x[k, l ] +
∂2im
∂2y[k, l ]
F−→ −(u2 + v2)IM[u, v ]
The Laplacian �lter is then implemented in the frequency domain by
H(u, v) = −(u2 + v2)
58
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
Laplacian in the frequency domain
Finally, to compute the Laplacian, we need :
1 to compute the Fourier transform of the picture;
2 to multiply the spectrum by −(u2 + v2);
3 to compute the inverse Fourier transform.
59
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
The cortex transform
The cortex transform is �rst described by A. Watson as the modeling of the neuralresponse of retinal cells to visual stimuli.
The cortex �lter in the frequency domain is
cortex[bi ,θi ](ρ, θ) = dombi× fanθi (θ)
Where,
bi and θi represent the frequency band and the index of orientation, respectively;
(ρ, θ) are polar coordinates.
The cortex transform decomposes the input image im[x , y ] into a set of subbandimages B[bi ,θi ][k, l ]:
B[bi ,θi ][k, l ] = F−1{cortex[bi ,θi ](ρ, θ)×F {im[x , y ]}
}
60
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionLow-pass �lters in spatial domainHigh-pass �lters in spatial domainDi�erentiation �lter in spatial domainFrequency domain �ltering
The cortex transform
Frequency responses of several cortex �lters (brightness represents gain for the givenspatial frequency).
Complete set of cortex �lters
61
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Introduction
1 Introduction
2 Point-to-point transformation
3 Linear �ltering (neighborhoodoperator)
4 Non Linear �lteringDe�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Conditional meanAnisotropic Kuwahara �lteringBilateral �ltering
5 Conclusion
62
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
De�nition
The most important drawback of the linear �ltering is that all pixels in the image aremodi�ed by the �ltering process.
To overcome this problem, non linear �ltering is used. It aims, for instance, to protectsome parts of the picture having particular features (edges...) or to remove data
without blurring the whole image (impluse noise).
Three kinds of �lters:
Rank �ltering;
Adaptive linear �ltering;
Morphological operators.
63
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Rank �ltering
Rank �lters are based on three steps:
1 Data selection, also calledwindowing;
2 Data ranking (in ascendant order);
3 1D linear weighting of the ordereddata list.
64
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Special case of a generalized rank �lter
If all weights of the linear �lter are null, except one in the median position. This �lteris called a median �lter.
IM[x , y ] = MED (im[xi , yi ]|[xi , yi ] ∈ V [x , y ])
V [x , y ] is the neighborhood (a set of N samples).
if the size of the neighborhood is odd, the output value is the median value;
if the size is even, the output value is the average of the two middles values.
The median �lter is very e�cient in �ltering signals corrupted by impulsive noise but itis not very e�cient in gaussian noise environment.
65
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Special case of a generalized rank �lter
However, when the number of the samples is large, the ordering procedure becomescumbersome.
Idea: the median �lter is taken over the outputs of several FIR substructures and thenumber of the substructures is much smaller than the number of the data samplesinside the �lter window.
IM[x , y ] = MED (y(1), . . . , y(m))
where, m is linear FIR �lters.
66
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Homomorphic �ltering
Homomorphic �ltering is a generalized technique for signal and image processing, in-volving a nonlinear mapping to a di�erent domain in which linear �lter techniques areapplied, followed by mapping back to the original domain.
In many case, we want to remove shading e�ects from an image. The objective is then
to enhance high frequencies;
to attenuate low frequencies (but �ne details have to be preserved).
Consider the following model of image formation:
im[x , y ] = i(x , y)︸ ︷︷ ︸illumination
× r(x , y)︸ ︷︷ ︸re�ection
The illumination component varies slowly and then a�ects low frequencies mostly;
The re�ection component varies faster and then a�ects the high frequenciesmostly.
67
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Homomorphic �ltering
What is the solution to separate LF and HF?
im[x , y ] = i [x , y ]× r [x , y ]
Fourier transform
IM[u, v ] = I [u, v ] ∗ R[u, v ]
Due to the convolution in the frequency domain, LF and HF from i [x , y ] and r [x , y ]are mixed together...
Too complex to �lter LF and HF in such condtion.
68
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Homomorphic �ltering
What is the solution to separate LF and HF? We can take the log !
im[x, y ] = i [x, y ]× r [x, y ]
log(im[x, y ]) = log(i [x, y ]× r [x, y ])
log(im[x, y ]) = log(i [x, y ]) + log(r [x, y ])
1 Take the log and apply the Fourier tansform to the new signal:
log(im[x, y ]) = log(i [x, y ]) + log(r [x, y ])
F (log(im[x, y ])) = F (log(i [x, y ])) + F (log(r [x, y ]))
Z [u, v ] = Ilog [u, v ] + Rlog [u, v ]
2 Filtering in the frequency domain: H[u, v ]
Z [u, v ]× H[u, v ] = Ilog [u, v ]× H[u, v ] + Rlog [u, v ]× H[u, v ]
3 Take the inverse Fourier transform and apply the exponential function:
F−1 (Z [u, v ]× H[u, v ]) = F−1 (Ilog [u, v ]× H[u, v ]
)+ F−1 (
Rlog [u, v ]× H[u, v ])
z[x, y ] = i [x, y ] + r [x, y ]
exp (z[x, y ]) = exp(i [x, y ]
)+ exp (r [x, y ])
69
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Homomorphic �ltering
H[u, v ] = γL +γH
1 +
[D0√u2+v2
]2nwith,
γL a parameter that a�ects low frequencies;
γH a parameter that a�ects high frequencies;
70
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Adaptive �ltering
An adaptive �lter is a �lter that self-adjusts its transfer function according to anoptimizing algorithm.
The goal is still to smooth the signal. However, we want to preserve edges...
Filtering by pixel grouping;
Conditional mean, Bilateral �ltering and mean shift �lter;
Di�usion.
71
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Conditional mean
IM[x , y ]︸ ︷︷ ︸Output
=∑
k∈V (x,y)
∑l∈V (x,y)
h(k, l)︸ ︷︷ ︸Filter coe�.
im[x − k, y − l ]︸ ︷︷ ︸Input
Principle: pixels in a neighbourhood are averaged only if they di�er from the centralpixel by less than a given threshold.
h(k, l) =
{1 if |IM[x − k, y − k]− IM[k, l ]| < TH
0 Otherwise.
Example with a neighbourhood equal (2× 3 + 1)(2× 3 + 1),TH = 32:
72
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Anisotropic Kuwahara �ltering
Method proposed by Kuwahara and adapted by Nagao in 1980.
Selection of the sub-domain thathas the minimum variance (9windows for Nagao);
Replace the value of the centralpixel by the average value of thesub-domain having the minimumvariance.
Example for a window 5× 5:
73
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Anisotropic Kuwahara �ltering
74
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Bilateral �ltering
The idea is to use a weighted �ltering but with an outlier rejection.Pixels that are very di�erent in intensity from the central pixel are weighted less eventhough they may be in close proximity to the central pixel.
This is applied as two Gaussian �lters at a localized pixel neighborhood:
IM[x , y ] =1∑
k,l c[x,y ][k, l ]s[x,y ][k, l ]
∑k,l
im[k, l ] c[x,y ][k, l ]s[x,y ][k, l ]︸ ︷︷ ︸h[k,l ]
One in the spatial domain named the domain �lter:
c[x,y ][k, l ] = exp
(−d([x , y ], [k, l ])
2σd
)where d is Euclidean distance.
One in the intensity space named the range �lter:
s[x,y ][k, l ] = exp
(−φ(im[x , y ], im[k, l ])
2σr
)where φ is a suitable measure of distance in intensity space.
75
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Bilateral �ltering
76
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
De�nitionRank �lteringHomomorphic �lteringAdaptive �ltering
Bilateral �ltering
Low contrast texture has been removed and edges are well preserved.
77
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Conclusion
1 Introduction
2 Point-to-point transformation
3 Linear �ltering (neighborhoodoperator)
4 Non Linear �ltering
5 Conclusion
78
IntroductionPoint-to-point transformation
Linear �ltering (neighborhood operator)Non Linear �ltering
Conclusion
Conclusion
Image Transformation: global to point; point to point; local to point;
Histogram;
Linear Filtering;
Non linear �ltering.
79