Digital Image Analysis and Processing CPE 0907544...Smoothing Spatial Filters yA smoothing...
Transcript of Digital Image Analysis and Processing CPE 0907544...Smoothing Spatial Filters yA smoothing...
Digital Image Analysis and ProcessingCPE 0907544CPE 0907544
Image Enhancement –Part IISpatial Filtering
Chapter 3S ti 3 4 3 7
D I d J f
Sections : 3.4-3.7
Dr. Iyad Jafar
OutlineIntroduction
Mechanics of Spatial Filtering
Correlation and Convolution
Linear Spatial FilteringLinear Spatial Filtering
Spatial filters for Smoothing
Spatial filters for Sharpening
S Nonlinear Spatial Filtering
Combining Spatial Enhancement Techniques Combining Spatial Enhancement Techniques 2
BackgroundFiltering is borrowed from the frequency domainprocessing and refers to the process of passing orprocessing and refers to the process of passing orrejecting certain frequency components
Highpass lowpass band-reject and bandpassHighpass, lowpass, band-reject , and bandpassfilters
Filtering is achieved in the frequency domain bydesigning the proper filter (Chapter 4)g g p p ( p )
Filtering can be done in the spatial domain also byusing filter masks (kernels, templates, or windows)
Unlike frequency domain filters, spatial filters canbe nonlinear !
3
Spatial Filtering MechanicsA spatial filter is characterized by
A rectangular neighborhood of size mxn (usually m and n are odd)A rectangular neighborhood of size mxn (usually m and n are odd)A predefined operation that is specified by the mask values at each position. Origin y
w(-1,-1) w(-1,0) w(-1,1)
y
w(0,-1) w(0,0) w(0,1)
w(1,-1) w(1,0) w(1,1) (x, y)
Neighbourhood
3x3 filter mask example
x Image f (x, y)
Spatial filtering OperationThe filter mask is centered at each pixel in the image and the outputpixel value is computed based on the operation specified by the
g f y
pixel value is computed based on the operation specified by themask4
Spatial Filtering MechanicsO i iOrigin
y
x3x3 Filter
w(-1,-1) w(-1,0) w(-1,1) f(x-1,y-1) f(x-1,y) f(x-1,y+1)
xImage f (x, y) xw(0,-1) w(0,0) w(0,1) f(x,y-1) f(x,y) f(x,y+1)
Filter Original Image Pixels
w(1,-1) w(1,0) w(1,1) f(x+1,y-1) f(x+1,y) f(x+1,y+11)
a b
g( x y ) w( s t ) f ( x s y t )= + +∑∑ ∑5 s a t b
g( x, y ) w( s,t ) f ( x s, y t )=− =−
= + +∑∑ ∑
Vector Representation of S ti l Filt iSpatial Filtering
The previous filtering equation can be written as The previous filtering equation can be written as
w1 w2 w31 1 2 2 mn mnR w z w z ... w z= + + +
w4 w5 w6
w7 w8 w9 1
mn
k kk
= w z ∑
If we represent the coefficients of the filter mask as a
w7 w8 w9 1k=
row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and the image pixels under the mask by z = [z1 z2 z3 z4 z5
] h h f l b z6 z7 z8 z9], then the filtering equation can be written as
TR wz=6
R wz=
Spatial CorrelationThe filtering operation just described is calledcorrelationIn 1-D
f wOrigin
0 0 0 1 0 0 0 0 1 2 3 2 81) Function and mask
1 2 3 2 8
0 0 0 1 0 0 0 02) Initial Alignment
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 03) Zero Padding for f by m-1
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
7 1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift
Spatial Correlation
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts
1 2 3 2 8
.6) Repeat .
..
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position
0 0 0 8 2 3 2 1 0 0 0 0
1 2 3 2 8
Full correlation result 0 0 0 8 2 3 2 1 0 0 0 0
8 Cropped correlation result
0 8 2 3 2 1 0 0
Spatial Convolution
A strongly related operation to correlation isconvolution (an important tool in linear systemstheory)
The mechanics of convolution is similar to thoseThe mechanics of convolution is similar to thoseof correlation, except that the filter mask isrotated by 180o before slidingrotated by 180 before sliding
9
Spatial Convolution
0 0 0 1 0 0 0 0
f wOrigin
1 2 3 2 81) Function and mask 0 0 0 1 0 0 0 0
0 0 0 1 0 0 0 0
1 2 3 2 81) Function and mask
8 2 3 2 12) Initial Alignment
after rotation ofmask
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 03) Zero Padding for f by m-1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift
8 2 3 2 14) Position after one shift
10
Spatial Convolution
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts
8 2 3 2 1
.6) Repeat .
..
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position
0 0 0 1 2 3 2 8 0 0 0 0
8 2 3 2 1
Full convolution result 0 0 0 1 2 3 2 8 0 0 0 0
11 Cropped convolutionresult
0 1 2 3 2 8 0 0
Extension to 2DExtension to 2D is straight forward
In convolution, the mask is flipped vertically and , pp yhorizontally Zero padding is done in both directions by m-1 and n-1
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 00 0 0 0 0 Zero padding by m 1 and n 1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 00 0 1 0 00 0 0 0 0
f
Zero padding by m-1 and n-1
0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0
1 2 3 9 8 7 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
1 2 34 5 67 8 9
9 8 76 5 43 2 1
w Mask rotation
12
0 0 0 0 0 0 0 0 0
Extension to 2D
9 8 7 0 0 0 0 0 06 5 4 0 0 0 0 0 0
1 2 3 0 0 0 0 0 04 5 6 0 0 0 0 0 0
3 2 1 0 0 0 0 0 00 0 0 0 0 0 0 0 0
7 8 9 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Initial position in convolution
0 0 0 0 0 0 0 0 0
Initial position in correlation
13
Extension to 2D
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 9 8 7 0 0 00 0 0 6 5 4 0 0 0
0 0 0 0 0 0 0 0 00 0 0 1 2 3 0 0 00 0 0 4 5 6 0 0 00 0 0 6 5 4 0 0 0
0 0 0 3 2 1 0 0 00 0 0 0 0 0 0 0 0
0 0 0 4 5 6 0 0 00 0 0 7 8 9 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0
Full convolution resultFull correlation result
14
Extension to 2D0 0 0 0 00 1 2 3 0
0 0 0 0 00 1 2 3 00 4 5 6 00 7 8 9 0
0 9 8 7 00 6 5 4 00 3 2 1 0
Cropped convolution result
0 7 8 9 00 0 0 0 0
Cropped correlation result
0 3 2 1 00 0 0 0 0
ppCropped correlation result
a b
Correaltion
( ) ( ) f ( )∑∑a b
Convolution
( ) ( ) f ( )∑∑s a t b
g( x, y ) w( s,t ) f ( x s, y t )=− =−
= + +∑∑s a t b
g( x, y ) w( s,t ) f ( x s, y t )=− =−
= − −∑∑
15
Treatment of Pixels at Edges
In the previous slides, we padded the image with zerosin both directions in order to compensate forunavailable values. x
Other approachesReplicate edge pixels
e e
Consider only available pixels that fall under the e
mask in the computation of the new values
e
Allow pixels to wrap around ee e
16y
Image f (x, y)
Treatment of Pixels at EdgesThe result depends on the used approachE l 3 6Example 3.6
Filtered Image: Z P ddiZero Padding
OriginalImage
Filtered Image: Replicate Edge PixelsImage Replicate Edge Pixels
Filtered Image: Wrap Around Edge Pixels
17
Wrap Around Edge Pixels
Smoothing Spatial FiltersA smoothing (averaging, blurring) filter replaces each i l i h h l f ll i l d h pixel with the average value of all pixels under the
mask
Uses of smoothing filtersThe smoothed image correspond to the gross details of the image or the low frequency content of the image C b d t d i it i h t i d ith Can be used to reduce noise as it is characterized with sharp transitions
However, smoothing usually result in degradation of ed e alit (sha p ess of the i a e)edge quality (sharpness of the image)
18
Smoothing Spatial FiltersCommon smoothing masks
1 1 1 1 2 1
1 1 1
1 1 1
X 1/9 2 4 2
1 2 1
X 1/16
Standard averaging mask Weighted average mask
NotesThe weighted average filter gives more weight to pixels near the centercenterIt is hard to see the visual difference between the processingresults of the two filters; however, the weighted average filtersummation is 16, which make it more attractive for computers
19
Smoothing Spatial FiltersExample: 3.7 Smoothing with different mask sizes. Notice the loss of details as the mask size increases
Original Smoothing by 3x3 Mask Smoothing by 5x5 Mask
20Smoothing by 9x9 Mask Smoothing by 15x15 Mask Smoothing by 35x35 Mask
Smoothing Spatial FiltersExample 3.8: smoothing highlights gross details. C ld b f l i idi b tt t ti Could be useful in providing better segmentation results
Original Image Smoothed Image Thresholded Image
21
Smoothing Spatial FiltersExample 3.9: Noise Reduction
22
Original Smoothed
Sharpening Spatial FiltersThe principle objective of sharpening is to highlighttransitions in intensity which usually correspond totransitions in intensity which usually correspond toedges in images; thus sharpening is the opposite ofsmoothingsmoothing
If we examine the smoothing operation we can thinkf it i t tiof it as integration
Thus to perform sharpening in the spatial domain, it isintuitive to use differentiation
In the following few slides we examine theIn the following few slides we examine thefundamental properties of the first and secondderivative when applied on different intensitypp ytransitions
23
Sharpening Spatial FiltersLets discuss differentiation for 1-D caseWe are concerned about the behavior of 1st and 2nd
derivatives in the following areas Constant intensityOnset and end of discontinuities (ramps and steps discontinuities)
Intensity ramps
Properties of 1st derivativeZero in areas of constant intensityNonzero at the onset of a step and intensity rampNonzero along intensity ramp
P 2 d dProperties 2nd derivativeZero in areas of constant intensityN t th t d d f t d i t it Nonzero at the onset and end of a step and intensity rampZero along intensity ramp 24
Sharpening Spatial FiltersDerivatives can be approximated as differences
1st derivative at x
1f f ( ) f ( )∂ 1f f ( x ) f ( x )x= + −
∂
2nd derivative at x
2
2 1 1 2f f ( x ) f ( x ) f ( x )∂= + + − −
∂25
2x∂
Sharpening Spatial FiltersInvestigation of derivatives behavior
26
Sharpening Spatial FiltersNotes
Examining the 1st and 2nd derivatives plots shows that allof their properties are satisfied
1st derivative produce thicker edges than 2nd derivatives
2nd derivative produce double edge separated by a zerocrossingcrossing
2nd derivative is commonly used in sharpening since it2nd derivative is commonly used in sharpening since ithas simpler implementation and finer edges
27
Sharpening Using 2nd
D i tiDerivativeThe second derivative (Laplacian) in 2-D is defined as
2 22
2 2f ff ∂ ∂
∇ = +
If we define
2 2fx y∂ ∂
If we define 2
2 1 1 2f f ( x , y ) f ( x , y ) f ( x, y )x
∂= + + − −
∂
2
2 1 1 2f f ( x, y ) f ( x, y ) f ( x, y )y
∂= + + − −
∂
Then the discrete second derivative can be approximated by
x∂ y∂
approximated by
2f∇ =1 1 1 1 4f ( x , y ) f ( x , y ) f ( x, y ) f ( x, y ) f ( x, y )+ + − + + + − −
28
Sharpening Using 2nd
D i tiDerivativeThe Laplacian can be implemented as a filter mask
0 1 0
1 -4 1
0 1 0
OrOr
1 1 1
1 -8 1
1 1 1
29
Sharpening Using 2nd
D i tiDerivativeComputing the Laplacian doesn’t produce a sharpenedp g p p pimage. However, grayish edge lines and discontinuitiessuperimposed on a dark background
It is common practice to scale the Laplacian image to [0,255] for better display30
Sharpening Using 2nd
D i tiDerivativeAlternatively, to obtain a sharpened image g(x,y), y p g g( y)subtract the Laplacian image from the original image
2( ) f ( ) f ( )2g( x, y ) f ( x, y ) f ( x, y )= − ∇
- =
Original Laplacian Sharpened
31
OriginalImage
LaplacianFiltered Image
SharpenedImage
Sharpening Using 2nd
D i iDerivativeThe two steps required to achieve sharpening can beThe two steps required to achieve sharpening can be
combined into a single filtering operation
ff 2)()( ∇)1()1([)( yxfyxfyxf −++−=
fyxfyxg 2),(),( ∇−=),1(),1([),( yxfyxfyxf −++−=)1,()1,( −+++ yxfyxf )()( yfyf
)],(4 yxf−),1(),1(),(5 yxfyxfyxf −−+−=)1()1( + ff )1,()1,( −−+− yxfyxf
32
Sharpening Using 2nd
DerivativeAs a filter mask the previous equation can be represented As a filter mask, the previous equation can be represented
as0 -1 0-1 5 -10 -1 0
33
Sharpening Using 1st DerivativeThe first derivative in image processing is implemented
as a gradient which is defined as a vectorg
⎥⎤
⎢⎡∂∂
⎤⎡f
G
⎥⎥⎥⎥
⎦⎢⎢⎢⎢
⎣∂∂∂=⎥
⎦
⎤⎢⎣
⎡=∇
yfx
GG
y
xf
The gradient points in the direction of greatest rate ofh l i ( )
⎥⎦⎢⎣∂y
change at location (x,y)The magnitude of the gradient is defined as
Or, approximately1
2 22f fM( x, y )⎡ ⎤⎛ ⎞∂ ∂⎛ ⎞⎢ ⎥= + ⎜ ⎟⎜ ⎟⎢ ⎥⎝ ⎠
34
( , y )x y⎜ ⎟⎜ ⎟∂ ∂⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦ | | x yf G G∇ ≈ +
Sharpening Using 1st DerivativeComputation of the gradient using Roberts
cross-gradient operatorsg pRoberts operators for computing Gx and Gy are shown
below. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5 in the original image
1 0 0 1Z1 Z2 Z3 -1 0
0 1
0 -1
1 0
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9
[ ]2
Horizontal OperatorZ7 Z8 Z9
Pixel z5 and its neighboursVertical Operator
[ ][ ]
29 5 9 5
28 6 8 6
x
y
G ( x, y ) z z z z
G ( x, y ) z z z z
= − ≈ −
= − ≈ −
35
[ ]8 6 8 6
9 5 8 6
y( , y )
M( x, y ) z z z z≈ − + −
Sharpening Using 1st DerivativeComputation of the gradient using Sobel
operatorspSobel operators for computing Gx and Gy are shown
below. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5
-1 0 1-1 -2 -1Z1 Z2 Z3 1 0 1
-2 0 2
1 0 1
1 2 1
0 0 0
1 2 1
Z1 Z2 Z3
Z4 Z5 Z6
Z7 Z8 Z9 -1 0 11 2 1
Mask to Compute Gx
Z7 Z8 Z9
Pixel z5 and its neighbours Mask to Compute Gy
7 8 9 1 2 3
3 6 9 1 4 7
2 2
2 2x
y
G ( x, y ) ( z z z ) ( z z z )
G ( x, y ) ( z z z ) ( z z z )
≈ + + − + +
= + + − + +
36
y
x yM( x, y ) G ( x, y ) G ( x, y )= +
Sharpening Using 1st DerivativeExample 3.10
Gradient is widely used in industrial inspection as ity pproduce thicker edges in the result, which make it easierfor machines to detect artifacts
37Optical Image for a contact lens Gradient Image
Nonlinear (Order-statistic) S ti l FiltSpatial Filters
Order-statistic filters are nonlinear filters whose response is based on ordering the pixels under the mask and then replacing the centre pixel with the value determined by the ranking result
Examples
Z1 Z2 Z3
Z4 Z5 Z6 Rankpminimum filtermaximum filter
Z7 Z8 Z9
median filter popular and useful in removing impulse or salt-and-pepper noise
New value
with less blurring than linear smoothing38
Nonlinear (Order-statistic) S ti l FiltSpatial Filters
Example 3.11p
Original Image Image After Image After
Note how the median filter has reduced the noise
With Noise Averaging Filter Median Filter
significantly with less blurring than smoothing filter39
Combining Spatial hEnhancements
It is very common to combine It is very common to combine different enhancement techniques in order to achieve techniques in order to achieve the desired goal
Example 3.12Example 3.12We want to enhance/sharpen
the bones in this imagegDirect application of the
Laplacian may result in noise amplification
40
Combining Spatial hEnhancements
Compare to enhancement by single method
Processed by a combination of
Sharpened by Laplacian
Histogram Equalization
Power-Law with gamma = 0 5
41
combination of methods
Laplacian Equalization gamma = 0.5
Combining Spatial E h tEnhancement
Enhancement by multiple methods
(a)
Laplacian filter of bone scan (a) (b)
Sharpened version of bone scan achieved by subtracting (a)
(c)y g ( )
and (b) Sobel filter of bone scan (a) (d)42
Combining Spatial E h tEnhancement
Result of applying a power law trans to
(h)
Sharpened image which is sum of (a) and (f)
power-law trans. to (g)
(g)The product of (c) and (e) which will be used as a mask
and (f)(f)
(e)
Image (d) smoothed witha 5*5 averaging filter
43
Combining Spatial E h tEnhancement
Compare the original and final imagesp g g
44
Readings
Read section 3 6 3 titled “Unsharp Masking Read section 3.6.3 titled Unsharp Masking and Highboost filtering”.
45
Related Matlab Functions
Check Matlab documentation for the following functions
2conv2
corr2
fillter2
imfilter
padarraypadarray
median
medfilt246