IP-L4 - 2010-Part2-Neighb
Transcript of IP-L4 - 2010-Part2-Neighb
![Page 1: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/1.jpg)
Image Enhancement
Part 2: Neighborhood operations
![Page 2: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/2.jpg)
Neighborhood operations
• Correlation ↔ pattern recognition
• Convolution ↔ Linear filtering– Edge detection– Denoising
![Page 3: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/3.jpg)
3
Correlation• Correlation
– Measures the similarity between two signals– Difference from convolution: no ‘minus’ signs in the formula
• the signals need only to be translated
pattern
database matching result
g...∑∑ ++=
k rtemplate rnkmhnmfnmC ],[],[),(
Application
![Page 4: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/4.jpg)
4
Convolution
response impulsefilter :],[image (output) filtered :],[image (input) original :],[
),(),(),(
],[],[],[],[],[,
nmhnmgnmf
jjHjjFjjG
nrmkhnmfnmhnmfnmg
filter
yxfilteryxyx
rkfilterfilter
ωωωωωω =
−−=∗= ∑
![Page 5: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/5.jpg)
5
Convolution and digital filtering
• Digital filtering consists of a convolution between the image and the impulse response of the filter, which is also referred to as convolution kernel.
• Warning: both the image and the filter are matrices (2D). – If the filter is separable, then the 2D convolution can be implemented as
a cascade of two 1D convolutions
• Filter types– FIR (Finite Impulse Response)– IIR (Infinite IR)
![Page 6: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/6.jpg)
6
Ideal interpolation digital filter|H(Ω)|
ΩΩN 2π-2π Ωc-Ωc
Digital LP filter (discrete time)
The boundary between the pass-band and the stop-band is sharp
The spectrum is periodic (the signal is sampled)
The repetitions are located at integer multiples of 2π
The low-pass filtered signal is still a digital signal, but with a different frequency content
The impulse response h[n] in the signal domain is discrete time and corresponds to the sinc[] function
Reconstruction LP filter (continuous time)
The boundary between the pass-band and the stop-band is sharp
The spectrum consists of one repetition only (the resulting signal is CT, ideally)
The low-pass filtered signal is a continuous time signal, that might have a different frequency content
The impulse response h(t) in the signal domain is continuous time and corresponds to the sinc() function
![Page 7: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/7.jpg)
7
Digital LP filtering|H(Ω)|
2 π-2π
|F(Ω)|
|F(Ω) H(Ω)|low-pass filtered (digital) signal
-Ωc Ωc
2 π2 π
2 π2 π
![Page 8: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/8.jpg)
8
LP and HP filteringLow-pass High-pass
n
averaging
n
|HLP(Ω)| |HHP(Ω)|
hLP[n] hHP[n]
Frequency domain
Signal domain
![Page 9: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/9.jpg)
9
Notation reminder
2 periodicity in Fourier domain for sampled signals (Ts=sample distance)
2 periodicity in FD after frequency normalization (Ts=1) relation among frequency and normalized frequency
u2
ss
s
s
T
T
u
πω
πω
π
=
Ω =
Ω =
Ω= nitary frequency (periodicity in frequency domain=1)
![Page 10: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/10.jpg)
10
Example: Chebyshev LP
Reminder:
Ω=2πu
this is u
![Page 11: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/11.jpg)
11
Example: Chebyshev HP
![Page 12: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/12.jpg)
12
Example: Chebyshev Impulse Response
LP HP
![Page 13: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/13.jpg)
13
Example: Chebyshev Step Response
![Page 14: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/14.jpg)
14
Example: filtered signal
The transfer function (or, equivalently, the impulse response) of the filter determines the characteristics of the resulting signal
![Page 15: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/15.jpg)
15
Switching to images
• Images are 2D digital signals (matrices) → filters are matrices
• Low-pass ↔ averaging (discrete interpolation) ↔ smoothing
• High-pass ↔ differentiation ↔ emphasize image details like lines, and, more in general, sharp variations of the luminance
• Band-pass: same as high pass but selecting a predefined range of spatial frequencies
Setting apart low-pass and high-pass image features is the ground of multi-resolution. It is advantageous for many tasks like contour extraction (edge detection), image compression, feature extraction for pattern recognition, image denoising etc.
![Page 16: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/16.jpg)
16
2D filtersLow-pass High-pass
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
111111111
91
lowpassh⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−−−−−
=111181111
highpassh
![Page 17: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/17.jpg)
17
Filtering in image domain
Filtering in image domain is performed by convolving the image with the filter kernel. This operation can be though of as a pixel-by-pixel product of the image with a moving kernel, followed by the sum of the pixel-wise output.
![Page 18: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/18.jpg)
18
Low-pass filtering: example
x =
f[m,n] hlowpass[m,n] g[m,n]
![Page 19: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/19.jpg)
19
Averaging
1111111111111111111111111
hlp=1/25
![Page 20: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/20.jpg)
20
Gaussian
hlp=0.0030 0.0133 0.0219 0.0133 0.00300.0133 0.0596 0.0983 0.0596 0.01330.0219 0.0983 0.1621 0.0983 0.02190.0133 0.0596 0.0983 0.0596 0.01330.0030 0.0133 0.0219 0.0133 0.0030
![Page 21: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/21.jpg)
21
Laplacian of Gaussian (LoG)0.0239 0.0460 0.0499 0.0460 0.02390.0460 0.0061 -0.0923 0.0061 0.04600.0499 -0.0923 -0.3182 -0.0923 0.04990.0460 0.0061 -0.0923 0.0061 0.04600.0239 0.0460 0.0499 0.0460 0.0239
hhp=
![Page 22: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/22.jpg)
22
Asymmetric LP
0 1 0
1 2 1
0 1 0hlp=1/6
![Page 23: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/23.jpg)
23
Asymmetric HP
-1 -1 -1
-1 8 -1
-1 -1 -1
hlp=
![Page 24: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/24.jpg)
24
Basic Highpass Spatial Filtering
• The filter should have positive coefficients near the center andnegative in the outer periphery:
Laplacian mask
Other Laplacian masks(normalization factor ismissing)
![Page 25: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/25.jpg)
25
Basic Highpass Spatial Filtering
• The sum of the coefficients is 0, indicating that when the filter is passing over regions of almost stable gray levels, the output of the mask is 0 or very small.
• The output is high when the center value differ from the periphery.
• The output image does not look like the original one.
• The output image depicts all the fine details
• Some scaling and/or clipping is involved (to compensate for possible negative gray levels after filtering).
![Page 26: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/26.jpg)
26
![Page 27: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/27.jpg)
Image enhancement by filtering
![Page 28: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/28.jpg)
28
Edge crispening
• Edge crispening is performed by a discrete convolution with a high pass filter
![Page 29: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/29.jpg)
29
Sharpening
• Goal: “improve” image quality
• Solutions– increase relative importance of details, by increasing the relative weight
of high frequency components• Increase a subset of high frequencies (non symmetric HP)• High-boost filter• Laplacian gradient
– The original image is assumed to be available
![Page 30: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/30.jpg)
30
Sharpening Filters
• To highlight fine detail or to enhance blurred details– Averaging filters smooth out noise but also blur details
• Sharpening filters enhance details
• May also create artifacts (amplify noise)
• Background: Derivative is higher when changes are abrupt
• Categories of sharpening filters– Basic highpass spatial filtering– High-boost filtering
![Page 31: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/31.jpg)
31
Sharpening
HPOriginal image +
Normalization(range=0,1)
Normalization(range=0,1)
Sharpened image
The normalization step subtracts the mean and scales the amplitude of the resulting image by dividing it for the dynamic range of the resulting image (graylevel values are now in the range 0-1)
For the sharpening to be visible, the sharpened and original images must then be displayed using the same set of graylevel values
De-normalization
(range=0,255)
![Page 32: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/32.jpg)
32
High boost
Examples
![Page 33: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/33.jpg)
33
High-boost
• Highpass filtered image = Original – lowpass filtered image
• If A is an amplification factor, then:
High-boost = A · original – I · lowpass= (A-I) · original + I · (original – lowpass)= (A-I) · original + I · highpass
Unsharp masking (if A=2I)
![Page 34: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/34.jpg)
34
Unsharp masking
original low resolution
c: weighting constant, usually ranging between 3/5 and 5/6
ringing – this can be reduced by using a HP filter with smooth cut-off
![Page 35: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/35.jpg)
35
Unsharp masking - ringing
![Page 36: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/36.jpg)
36
(1) (2)
(3) = (1) - (2)(AI-1) (1) + (3)
Signal Low- pass
High-pass
Unsharp Masking and Sharpening operation
![Page 37: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/37.jpg)
37
High-boost Filtering
• A=1 : standard highpass result
• A > 1 : the high-boost image looks more like the original with a degree of edge enhancement, depending on the value of A.
w=9A-1, A≥1
A > 1 Unsharp masking
![Page 38: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/38.jpg)
38
![Page 39: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/39.jpg)
39
![Page 40: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/40.jpg)
40
Sharpening: asymmetric HP
![Page 41: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/41.jpg)
41
Sharpening: the importance of phase
![Page 42: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/42.jpg)
42
Phase
Signal
Edges
Enhanced
HP filter f
![Page 43: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/43.jpg)
43
Phase
Signal
Edges
Enhanced
HP filter -f
![Page 44: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/44.jpg)
44
Sharpening: the importance of phase
![Page 45: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/45.jpg)
45
Sharpening: the importance of phase
![Page 46: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/46.jpg)
46
Sharpening: the importance of phase
![Page 47: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/47.jpg)
47
Back to the natural image
![Page 48: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/48.jpg)
48
Edge crispening by statistical differencing
[ ] [ ][ ]
,,
,F j k
G j kD j k
=F[j,k]: original imageG[j,k]: enhanced imageD[j,k]: local standard deviation
[ ] [ ]( )
[ ] [ ]
12 2
2
1[ , ] , ,
2 11, ,
j w j w
m j w n j w
j w j w
m j w n j w
D j k F m n M m nW
W w
M m n F m nW
+ +
= − = −
+ +
= − = −
⎡ ⎤= −⎢ ⎥
⎢ ⎥⎣ ⎦= +
=
∑ ∑
∑ ∑ local mean value
The enhanced image is increased in amplitude with respect to the original atpixels that deviate significantly from their neighbors, and is decreased in relative amplitude elsewhere.
![Page 49: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/49.jpg)
49
Statistical differencing: Wallis operator
• The enhanced image is forced to a form with desired first- and second-order moments.
• Wallis operator is defined by
– where • Md and Dd represent desired average mean and standard deviation factors,• Amax is a maximum gain factor that prevents overly large output values when
D[j,k] is small• 0≤p≤1 is a mean proportionality factor controlling the background flatness of
the enhanced image
![Page 50: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/50.jpg)
50
Statistical differencing: Wallis operator
• Alternative expression
– where A[j,k ]is a spatially dependent gain factor and B[j,k] is a spatially dependent background factor
– It is convenient to specify the desired average standard deviation Ddsuch that the spatial gain ranges between maximum Amax and minimum Amin limits. This can be accomplished by setting Dd to the value
Dmax is the maximum value of D[j,k]
![Page 51: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/51.jpg)
51
Wallis enhancement
![Page 52: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/52.jpg)
52
Wallis enhancement
![Page 53: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/53.jpg)
53
Wallis enhancement
![Page 54: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/54.jpg)
54
Color image enhancement
• Usually performed after transformation to an opposed channels model featuring lower correlation– Lab, Luv, YIQ, YCbCr
• Often enhancing the luminance, or lightness, component is enough– Low-pass behaviour of color vision
• Because of the high-spatial-frequency response limitations of human vision, edge crispening of the chroma or chrominance components may not be perceptible
• The channels are processed independently and then “recombined”after enhancement– Care must be taken to preserve the average value of the three channels
in every point to avoid color artifacts
![Page 55: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/55.jpg)
55
Color image sharpening1. RGB to YCbCr2. Smoothing the three components, respectively3. Unsharpening of the lightness (Y) component4. Reconstruction
-0.1667 -0.6667 -0.1667-0.6667 4.3333 -0.6667-0.1667 -0.6667 -0.1667
hu=
unsharpening filter
low pass
1 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 11 1 1 1 1
h=0,04 x
![Page 56: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/56.jpg)
56
Example
![Page 57: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/57.jpg)
57
smoothed Y
![Page 58: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/58.jpg)
58
smoothed chrominances
![Page 59: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/59.jpg)
59
all smoothed
![Page 60: IP-L4 - 2010-Part2-Neighb](https://reader031.fdocuments.in/reader031/viewer/2022012423/61775dabf5f66a57481a3a95/html5/thumbnails/60.jpg)
60
sharpened Y