7/27/2019 Smoothing Techniques in Image Processing[1]
1/59
Smoothing Techniques in ImageProcessing
Prof. PhD. Vasile Gui
Polytechnic University
of Timisoara
7/27/2019 Smoothing Techniques in Image Processing[1]
2/59
7/27/2019 Smoothing Techniques in Image Processing[1]
3/59
Introduction
Why do we need image
smoothing?
What is image and what
is noise?
Frequency spectrum
Statistical properties
7/27/2019 Smoothing Techniques in Image Processing[1]
4/59
Brief Review of Linear Operators
[Pratt 1991]
Generalized 2D linear operator
Separable linear operator:
Space invariant operator:
Convolution sum
1
0
1
0
),(),;,(),(M
j
N
k
kjfnmkjOnmg
);();(),;,( nkOmjOnmkjO CR
1
0
1
0
),();();(),(M
j
N
k
CR kjfnkOmjOnmg
),(),(),;,( knjmHknjmOnmkjO
1
0
1
0
),(),(),(M
j
N
k
kjfknjmhnmg
7/27/2019 Smoothing Techniques in Image Processing[1]
5/59
Brief Review of Linear Operators
Geometrical
interpretationof 2D
convolution
h0
0
h01h02
h10
h20h21h22
h11h12
h0
0
h01h02
h10
h20h21h22
h11h12
h0h01h02
h10
h20
h21
h22
h11h12h0
0
h01h02
h10
h20h21h22
h11
h12
N+L-1
N
N-
L+1
h0
0
h01h02h10
h20h21h22
h11h12
L-1
2L-1
2
L-1
2 L-1
2H
G
F
7/27/2019 Smoothing Techniques in Image Processing[1]
6/59
Matrix formulation
Unitary transform
Separable transform
Transform is invertible
Brief Review of Linear Operators
TRCFAAT
IfAfAtAf*T*T
*T1AA
Aft
7/27/2019 Smoothing Techniques in Image Processing[1]
7/59
Brief Review of Linear Operators
Basis vectors are orthogonal
Inner product and energy are conserved
Unitary transform: a rotation in N-dimensional vector
space
lk
lk
lk ,0
,1
),(lT*
k aa
2T*T*T*T*
*T*T2
||f||ffIffAfAf
(Af)(Af)tt||t||
7/27/2019 Smoothing Techniques in Image Processing[1]
8/59
Brief Review of Linear Operators
2D vector space interpretation
of a unitary transform
x1
x2
12
7/27/2019 Smoothing Techniques in Image Processing[1]
9/59
Brief Review of Linear Operators
DFT unitary matrix
A
1
0 0 0 0
0 1 2 1
0 2 4 2 1
0 2 1 1 2
N
W W W W W W W W
W W W W
W W W W
N
N
N N N
( )
( ) ( )
)2
sin()2
cos(}2
exp{
N
i
NN
iWN
7/27/2019 Smoothing Techniques in Image Processing[1]
10/59
Brief Review of Linear Operators
1D DFT
2D DFT
1,...,1,0,}2exp{)(1)(1
0
NuunNinf
Nut
N
n
1,...,1,0,1,...,1,0
)}(2exp{),(
1
),(
1
0
1
0
NvMu
nN
v
mM
u
inmfMNvut
M
m
N
n
7/27/2019 Smoothing Techniques in Image Processing[1]
11/59
Brief Review of Linear Operators
Circular convolution theorem
Periodicity off,h and g with Nare assumed
t u vN
t u v t u vg f h( , ) ( , ) ( , )1
1
0
1
0),(),(),(
N
j
N
kkjfknjmhnmg
7/27/2019 Smoothing Techniques in Image Processing[1]
12/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
1
1
1
1*111
1
111
1
111
111
12
LLLh
7/27/2019 Smoothing Techniques in Image Processing[1]
13/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Separable
Can be computed recursively, resulting inroughly 4 operations per pixel
L pixels
+
m,n
m,n+1
7/27/2019 Smoothing Techniques in Image Processing[1]
14/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Optimality properties
Signal and additive white noise
Noise variance is reduced Ntimes
g f n f n
1 1 1
1 1 1N N Nk k
k
N
kk
N
kk
N
( ) .
z n
1
1N kk
N
.
2
1 1
2
2
1 12
1 12
T2
1
),(
1
}E{1
}E{1
}E{
N=klN
nnN
nnN
zz
N
k
N
l
N
k
N
l
kl
N
k
N
l
klz
7/27/2019 Smoothing Techniques in Image Processing[1]
15/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Unknown constant signal plus noise
Minimize MSE of the estimation g:
N
k
k gfg1
22)()(
0)(2
g
g
N
k
kfN
g1
1
7/27/2019 Smoothing Techniques in Image Processing[1]
16/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
i.i.d. Gaussian signal with unknown mean.
Given the observed samples, maximize
Optimal solution: arithmetic mean
}2
)(exp{.)|( 2
2
fctfp
N
k
N
k
kk fctfp1 1
2
2})(
2
1exp{.)|(
7/27/2019 Smoothing Techniques in Image Processing[1]
17/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Frequency response:
Non-monotonically decreasing with frequency1
9
1 1 1
1 1 1
1 1 1
1
3111
1
3
1
1
1
[ ] h hx y
t u h h n iN
unx xn N
N
( ) ( ) ( ) exp{ }( )/
( )/
0 21 2
1 2
)2
cos(3
2
3
1)( u
Nut
7/27/2019 Smoothing Techniques in Image Processing[1]
18/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
An example
8 5 8 8 5 8
8 5 8 8 5 8
8 5 8 8 5 8
1
9
1 1 1
1 1 1
1 1 1
7 7 7 7 7 7
7 7 7 7 7 7
7 7 7 7 7 7
8 5 8 5 8 5
8 5 8 5 8 5
8 5 8 5 8 5
1
9
1 1 1
1 1 1
1 1 1
6 7 6 7 6 7
6 7 6 7 6 7
6 7 6 7 6 7
7/27/2019 Smoothing Techniques in Image Processing[1]
19/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Image smoothed with
33, 55, 99
and 11 11 box filters
7/27/2019 Smoothing Techniques in Image Processing[1]
20/59
Linear Image smoothing techniques
Box filters. Arithmetic mean LL operator
Original Lena imageLena image filtered with
5x5 box filter
7/27/2019 Smoothing Techniques in Image Processing[1]
21/59
Linear Image smoothing techniques
Binomial filters [Jahne 1995]
Computes a weighted average of pixels in
the window Less blurring, less noise cleaning for the
same size
The family of binomial filters can be defined
recursively
The coefficients can be found from (1+x)n
7/27/2019 Smoothing Techniques in Image Processing[1]
22/59
Linear Image smoothing techniques
Binomial filters. 1D versions
112
11b
11 bbb *1214
12
1111 bbbbb ***1464116
14
111111 bbbbbbb *****161520156164
16
As size increases, the shape of the filter is closer to a Gaussian one
7/27/2019 Smoothing Techniques in Image Processing[1]
23/59
Linear Image smoothing techniques
Binomial filters. 2D versions
121
242
121
16
1
1
2
1
4
1*121
4
12b
14641
41624164
62436246
41624164
14641
2561
1
4
6
4
1
161*14641
1614b
7/27/2019 Smoothing Techniques in Image Processing[1]
24/59
Linear Image smoothing techniques
Binomial filters. Frequency response
121
4
12
b )
2cos(
2
1
2
1)( u
N
ut
1464116
14
b 2)]2
cos(2
1
2
1[)( u
Nut
Monotonically decreasing
with frequency
7/27/2019 Smoothing Techniques in Image Processing[1]
25/59
Linear Image smoothing techniques
Binomial filters. Example
Original Lena image Lena image filtered
with binomial 5x5 kernel
Lena image filtered
with box filter 5x5
7/27/2019 Smoothing Techniques in Image Processing[1]
26/59
Linear Image smoothing techniques
Binomial and box filters. Edge blurring comparison
Linear filters have to compromise smoothing
with edge blurring
Step edge
Result of size L
box filter
L
Size L binomial
7/27/2019 Smoothing Techniques in Image Processing[1]
27/59
Nonlinear image smoothingThe median filter [Pratt 1991]
Block diagram
2
1
Nm
f1
f2
fN
.
.
.
Order
samples
f(1)f(2)
f(N)
.
.
.
selectf(m)
median
N is odd
)()3()2()1( Nffff
7/27/2019 Smoothing Techniques in Image Processing[1]
28/59
Nonlinear image smoothingThe median filter
Numerical example
67
3 7 8
2 3
4 6 7
2, 3, 3, 4, 6, 7, 7, 7, 8
7/27/2019 Smoothing Techniques in Image Processing[1]
29/59
Nonlinearity
Nonlinear image smoothingThe median filter
median{f1 + f2} median{ f1} + median{ f2}.
However:
median{ c f } = c median{ f},
median{ c + f } = c + median{ f}.
The filterselects a sample from the window, doesnot average
Edges are better preserved than with liner filters
Best suited for salt and pepper noise
7/27/2019 Smoothing Techniques in Image Processing[1]
30/59
Nonlinear image smoothingThe median filter
Noisy image 5x5 median filtered 5x5 box filter
7/27/2019 Smoothing Techniques in Image Processing[1]
31/59
Nonlinear image smoothingThe median filter
Optimality
Grey level plateau plus noise. Minimize sum of
absolute differences:
Result:
If 51% of samples are correct and 49% outliers,
the median still finds the right level!
N
k
k gfg1
||)(
)(321 },,,,{ mN fffffmediang
7/27/2019 Smoothing Techniques in Image Processing[1]
32/59
Nonlinear image smoothingThe median filter
Caution: points, thin lines and corners are erased by the median filter
Test images
Results of 33 pixel median filter
7/27/2019 Smoothing Techniques in Image Processing[1]
33/59
Nonlinear image smoothingThe median filter
Cross shaped window can correct some of the problems
Results of the 9 pixel cross shaped window median filter
7/27/2019 Smoothing Techniques in Image Processing[1]
34/59
Nonlinear image smoothingThe median filter
Implementing the median filter
Sorting needs O(N2
) comparisons Bubble sort
Quick sort
Huang algorithm (based on histogram)
VLSI median
7/27/2019 Smoothing Techniques in Image Processing[1]
35/59
Nonlinear image smoothingThe median filter
VLSI median block diagram
a
b
Min(a,b)
Max(a,b)
x(1)
x(2)
x(3)
x(4)
x(5)
x(6)
x(7)
x1
x2
x3
x4
x5
x6
x7
7/27/2019 Smoothing Techniques in Image Processing[1]
36/59
Nonlinear image smoothingThe median filter
Color median filter There is no natural ordering in 3D (RGB) color
space Separate filtering on R,G and B components does
not guarantee that the median selects a truesample from the input window
Vector median filter, defined as the sampleminimizing the sum of absolute deviations from allthe samples
Computing the vector median is very timeconsuming, although several fast algorithms exist
7/27/2019 Smoothing Techniques in Image Processing[1]
37/59
Nonlinear image smoothingThe median filter
Example of colormedian filtering
5x5 pixels window
Up: original image
Down: filtered image
7/27/2019 Smoothing Techniques in Image Processing[1]
38/59
Nonlinear image smoothingThe weighted median filter
The basic idea is to give higher weight to some samples, according to
their position with respect to the center of the window
Each sample is given a weight according to its spatial position in the
window.
Weights are defined by a weighting mask
Weighted samples are ordered as usually
The weighted median is the sample in the ordered array such that
neither all smaller samples nor all higher samples can cumulate more
than 50% of weights.
If weights are integers, they specify how many times a sample is
replicated in the ordered array
7/27/2019 Smoothing Techniques in Image Processing[1]
39/59
Nonlinear image smoothingThe weighted median filter
Numerical example for the weighted median filter
67
3 7 8
2 3
4 6 7
2, 2, 3, 3, 3, 3, 4, 6, 6, 7, 7, 7, 7, 7, 8
3
1 1
1 1
2 2
2
2
7/27/2019 Smoothing Techniques in Image Processing[1]
40/59
Nonlinear image smoothingThe multi-stage median filter
Better detail preservation
mi = median( Ri ), i =1,2,3,4.
Result = median( m1,m2,m3,m4,m5 )
R1
R2
R3
R4
m5
7/27/2019 Smoothing Techniques in Image Processing[1]
41/59
Nonlinear image smoothingRank-order filters (L filters)
Block diagram
sum
y
f(1)f1
f2
fN
.
.
.
ordering
f(2)
f(N)
.
.
.
a1
a2
aN
weights
y= a1f(1) + a2f(2)+...+ aNf(N)akk
N
1
1
7/27/2019 Smoothing Techniques in Image Processing[1]
42/59
7/27/2019 Smoothing Techniques in Image Processing[1]
43/59
Nonlinear image smoothingRank-order filters (L filters)
Optimality
( ) | |( )f f fk rk
N
1
Minkovski distance
Case r= 1: best estimator is median.
Case r= 2, best estimator is arithmetic mean.
Case r, best estimator is mid-range.
7/27/2019 Smoothing Techniques in Image Processing[1]
44/59
Nonlinear image smoothingRank-order filters (L filters)
Alpha-trimmed mean filter
otherwiseQmkQmforQak
,0
),12/(1
= (2Q + 1) / N, defines de degree of averaging
= 1 corresponds to arithmetic mean
= 1 / N corresponds to the median filter
Properties: in between mean and median
7/27/2019 Smoothing Techniques in Image Processing[1]
45/59
Nonlinear image smoothingRank-order filters (L filters)
Median of absolute differences trimmed mean Better smoothing than the median filter and good edge
preservation
},,,,{ 3211 NffffmedianM
NiMfmedianM i ,...,2,1|},{| 12
}||:{ 21 MMffaverageoutput ii M2 is a robust estimator of the variance
7/27/2019 Smoothing Techniques in Image Processing[1]
46/59
Nonlinear image smoothingRank-order filters (L filters)
k nearest neighbour (kNN) median filter
Median of the k grey values nearest by rankto the central pixel
Aim: same as above
7/27/2019 Smoothing Techniques in Image Processing[1]
47/59
Kuwahara type filtering Form regions Ri
Compute mean, mi and variance,Si for each region.
Result = mi : mi < mj for all j
different from i, i,e. themean of
the most homogeneous region
Matsujama & Nagao
Nonlinear image smoothingSelected area filtering [Nagao 1980]
reg 1 reg 2
reg 3
reg 4
reg 5 reg 6
reg 7reg 8
7/27/2019 Smoothing Techniques in Image Processing[1]
48/59
Nonlinear image smoothingConditional mean
Pixels in a neighbourhood are averaged only if they
differ from the central pixel by less than a given
threshold:
otherwise
thnmflnkmfiflkh
lnkmflkhnmgL
Lk
L
Ll
,0
|),(),(|,1),(
),,(),(),(
L is a spacescale parameter and th is a range scale parameter
7/27/2019 Smoothing Techniques in Image Processing[1]
49/59
Nonlinear image smoothingConditional mean
Example with L=3,
th=32
7/27/2019 Smoothing Techniques in Image Processing[1]
50/59
Nonlinear image smoothingBilateral filter [Tomasi 1998]
Space and range are treated in a similar way
Space and range similarity is required for the averaged pixels
Tomasi and Manduchi [1998] introduced soft weights to penalize the
space and range dissimilarity.
)),(),((),(),( nmflnkmfrlkslkh
k l
k l
lkhK
lnkmflkhK
nmg
),(
,),(),(1
),(
s() and r() are space and range similarity functions (Gaussian
functions of the Euclidian distance between their arguments).
7/27/2019 Smoothing Techniques in Image Processing[1]
51/59
Nonlinear image smoothingBilateral filter
The filter can be seen as weighted averaging in the
joint space-range space (3D for monochromatic
images and 5D x,y,R,G,B - for colour images) The vector components are supposed to be properly
normalized (divide by variance for example)
The weights are given by:
));K(d()(
}||||exp{)(
2
sh
sh
c
c
xxx
xxx
7/27/2019 Smoothing Techniques in Image Processing[1]
52/59
Nonlinear image smoothingBilateral filter
Example of Bilateral filtering
Low contrast texture has been removed
Yet edges are well preserved
7/27/2019 Smoothing Techniques in Image Processing[1]
53/59
Nonlinear image smoothingMean shift filtering [Comaniciu 1999, 2002]
Mean shift filtering replaces each pixels value with the most probable local value, found
by a nonparametric probability density estimation method.
The multivariate kernel density estimate obtained in the point x with the kernel K(x) and
window radius r is:
For the Epanechnikov kernel, the estimated normalized density gradient is proportional
to the mean shift:
d
inii R xx ...1}{
n
id r
Knr
f1
1)( i
xxx
)(
21
)()(
)(
2 xxi
x
xxxx
x
ri S
rn
Mf
f
d
r
S is a sphere of radius r, centered on x and nx is the number of samples inside the sphere
7/27/2019 Smoothing Techniques in Image Processing[1]
54/59
Nonlinear image smoothingMean shift filtering
The mean shift procedure is
a gradient ascent method to
find local modes (maxima)
of the probability density
and is guaranteed to
converge.
Step1: computation of the
mean shift vector Mr(x). Step2: translation of the
window Sr(x) by Mr(x).
Iterations start for each pixel
(5D point) and tipically
converge in 2-3 steps.
7/27/2019 Smoothing Techniques in Image Processing[1]
55/59
Nonlinear image smoothingMean shift filtering
Example1.
7/27/2019 Smoothing Techniques in Image Processing[1]
56/59
Nonlinear image smoothing
Mean shift filtering
Detail of a 24x40
window from the
cameraman imagea) Original data
b) Mean shift paths for
some points
c) Filtered data
d) Segmented data
7/27/2019 Smoothing Techniques in Image Processing[1]
57/59
Nonlinear image smoothing
Mean shift filtering
Example 2
7/27/2019 Smoothing Techniques in Image Processing[1]
58/59
Nonlinear image smoothing
Mean shift filtering
Comparison to bilateral filtering Both methods based on simultaneous processing of both the
spatial and range domains
While the bilateral filtering uses a static window, the mean shift
window is dynamic, moving in the direction of the maximum
increase of the density gradient.
7/27/2019 Smoothing Techniques in Image Processing[1]
59/59
REFERENCES D. Comaniciu, P. Meer: Mean shift analysis and applications. 7th
International Conference on Computer Vision, Kerkyra, Greece, Sept.1999, 1197-1203.
D. Comaniciu, P. Meer: Mean shift: a robust approach toward featurespace analysis. IEEE Trans. on PAMI Vol. 24, No. 5, May 2002, 1-18.
M. Elad: On the origin of bilateral filter and ways to improve it. IEEEtrans. on Image Processing Vol. 11, No. 10, October 2002, 1141-1151.
B. Jahne: Digital image processing. Springer Verlag, Berlin 1995.
M. Nagao, T. Matsuiama: A structural analysis of complex aerialphotographs. Plenum Press, New York, 1980.
W.K. Pratt: Digital image processing. John Wiley and sons, New York1991
C. Tomasi, R. Manduchi: Bilateral filtering for gray and color images.Proc. Sixth Intl. Conf. Computer Vision , Bombay, 839-846.
Top Related