ELE 488 F06
ELE 488 Fall 2006Image Processing and Transmission
09-26-06
Linear 2-D Image Filtering
1-D discrete convolution 2-D discrete convolution 2-D spatial masks Mask filtering Mask filtering and 2-D convolution
Spatial Averaging, Blurring, Image Sharpening
Edge Map - gradient
9/26/06
ELE 488 F06
Signal Processing
output signal y[n]
input signal x[n]
y[n] = Σ h[k] x[n – k] , convolution
Processor
LTI system
Example: h[ ] [ 0.25 0.5 0.25 ],
y[n] = 0.25 x[n–1] + 0.5 x[n] + 0.25 x[n–1]
weighted sum of input
Commutative: A * B = B * A
Associative: (A * B ) * C = A * (B * C)
Distributive: A * (B + C) = A * B + A * C
ELE 488 F06
2D Signal Processing (image)
0.250.25
0.250.25
output signal y[m,n]
input signal x[m,n]
y[m,n] = Σ h[k, j] x[m – k, n – j] , 2D convolution
Processor
LSI system
Example: h[ ]:
y[m,n] = 0.25 x[m,n] + 0.25 x[m–1, n] + 0.25 x[m, n–1]
+ 0.25 x[m–1, n–1]
weighted sum of 4 pixels
ELE 488 F06
-1 0 10 1
-1 0 1 0 -1
0 -1 0
0
1
0 1
-1 0
1 0 -1
1 0 -1
1 0 -1
0 -1 0
0
1
0 1
-1 0
1 0 -1
1 0 -1
1 0 -1
0 -1 0
0
1
0 1
-1 0 1 0 -1
0 -1 0
0
1
0 1
-1 0 1 0 -1
0 -1 0
0
1
1 0 -1
Rotate 180
Do same for next row
xy
h
Example: y[m,n] = Σ h[k, j] x[m – k, n – j]
ELE 488 F06
Spatial Masks
Spatial Mask
Finite, 2-D, real valued, array.
One element is designated as (0,0).
Used to compute the weighted sum of an input image, where weights are specified by the mask.
Usually has small support 2x2, 3x3, 5x5, 7x7
Examples:
These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
ELE 488 F06
Applying a 2X2 Spatial Mask to an Image (1)
1/4 1/4
1/4 1/4
Input Image Output Image
1/4 1/4
1/4 1/4
ELE 488 F06
Applying a 2X2 Spatial Mask to an Image (2)
1/4 1/4
1/4 1/4
Input Image Output Image
Shift mask over input image = Shift input image under mask
1/4 1/4
1/4 1/4
ELE 488 F06
Applying a 2X2 Spatial Mask to an Image (3)
1/4 1/4
1/4 1/4
Input Image Output Image1/4 1/4
1/4 1/4
ELE 488 F06
Applying a 2X2 Spatial Mask to an Image (4)
1/4 1/4
1/4 1/4
Input Image Output Image1/4 1/4
1/4 1/4
Edge effect: what to do?
ELE 488 F06
Applying a 2X2 Spatial Mask to an Image (5)
1/4 1/4
1/4 1/4
Input Image Output Image1/4 1/4
1/4 1/4
ELE 488 F06
Applying a 3X3 Spatial Mask to an Image (1)
Input Image Output Image
-1/4 -1/4
-1/4
-1/4
1
0 0
00
-1/4 -1/4
-1/4
-1/4
1
0 0
00mask
ELE 488 F06
Applying a 3X3 Spatial Mask to an Image (2)
Input Image Output Image
-1/4 -1/4
-1/4
-1/4
1
0 0
00
-1/4 -1/4
-1/4
-1/4
1
0 0
00mask
ELE 488 F06
Applying a 3X3 Spatial Mask to an Image (3)
Input Image Output Image
-1/4 -1/4
-1/4
-1/4
1
0 0
00
-1/4 -1/4
-1/4
-1/4
1
0 0
00mask
ELE 488 F06
Spatial Masks
Spatial Mask
Finite, 2-D, real valued, array.
One element is designated as (0,0).
Used to compute the weighted sum of an input image, where weights are specified by the mask.
Usually has small support 2x2, 3x3, 5x5, 7x7
Examples:
These examples are spatial averaging masks. Used for image smoothing, LPF before subsampling (anti-aliasing), etc.
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
ELE 488 F06
Spatial Averaging Masks
• For softening, noise smoothing, LPF before subsampling(anti-aliasing), etc.
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
UM
CP
EN
EE
40
8G
/63
1 S
lide
s (c
rea
ted
by
M.W
u &
R.L
iu ©
20
02
/20
04
)
ELE 488 F06
Spatial Averaging Masks
1/4 1/4
1/4 1/4
0 1
0
1
UM
CP
EN
EE
40
8G
/63
1 S
lide
s (c
rea
ted
by
M.W
u &
R.L
iu ©
20
02
/20
04
)
ELE 488 F06
Spatial Averaging Masks
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
UM
CP
EN
EE
40
8G
/63
1 S
lide
s (c
rea
ted
by
M.W
u &
R.L
iu ©
20
02
/20
04
)
ELE 488 F06
Spatial Averaging Masks
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
UM
CP
EN
EE
40
8G
/63
1 S
lide
s (c
rea
ted
by
M.W
u &
R.L
iu ©
20
02
/20
04
)
ELE 488 F06
Spatial Averaging Masks
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
UM
CP
EN
EE
40
8G
/63
1 S
lide
s (c
rea
ted
by
M.W
u &
R.L
iu ©
20
02
/20
04
)
ELE 488 F06
The Equations
Let x(m,n) denote the input image, M(m,n) denote the mask and y(m,n) denote the output image. Then for each pixel (m,n) in the output image:
where the indices i and j range over the extent of the mask. This corresponds to shifting the image under the mask. This equation assumes values are provided in the edge regions.
1/4 1/4
1/4 1/4
0 1
0
1
1/9 1/9
1/9 1/9
-1 0 1
-1
0
1
1/9
1/9
1/9 1/9 1/9
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
U1
L1i
U2
L2j
j)M(i,j)ni,x(mn)y(m,
ELE 488 F06
The Equations, cont.
U1m
L1mi
U2m
L2nj
n)-jm,-M(ij)x(i,n)y(m,
Alternatively we can shift the mask over the image. In this case, for each pixel (m,n) in the output image:
If we pad the input image and the mask with zeros so that they are both of infinite extent, then we can write:
j)M(i,j)ni,x(mn)jm,M(ij)x(i,n)y(m,-i j-i j
ELE 488 F06
Point Spread Function
• For linear shift invariant (LSI) processing, the impulse response h[m,n] is called the point spread function :
• Delta function (point) in, point spread function out
ELE 488 F06
2-D Masking = Convolution with a PSF
• Mask Filter:
• Convolution:
U1
L1i
U2
L2j
U1
L1i
U2
L2j
n)-jm,-M(ij)x(i,j)M(i,j)ni,x(mn)y(m,
U1
L1i
U2
L2j
U1
L1i
U2
L2j
j)-ni,-h(mj)x(i,j)h(i,j)-ni,-x(mn)y(m,
Applying a mask filter M to an image is equivalent to convolution with the PSF h with: M(I, j) = h(– i, – j)
ELE 488 F06
The transformation is equivalent to a rotation by 180 degrees. Or flip on both indeces.
PSF to Mask
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
0 1/8
1/8 1/2
-1 0 1
-1
0
1
0
1/8
0 1/8 0
-1 0
-2 0
1
2
-1 0 1
1 0
2 0
-1
-2
1 0 -1
-1 0 1
1 0 -1
0 1
-1 0
0 -1
1 0
Notation:
Hence:
ELE 488 F06
Filtering an Image with a Mask
Filtering an image with a mask is a linear shift-invariant operation. Hence:
Masks can be applied in any order
Masks applied sequentially can first be combined into one mask. Exercise:
-1 0 10 1
-1 0 1 0 -1
0 -1 0
0
1
ELE 488 F06
LPF, HPF, & BPF
• Spatial averaging filters (usually positive entries in mask) are Low Pass Filters (LPF)
• HPF and BPF can be constructed from LPF– High-pass filter
• hHP(m,n) = (m,n) – hLP(m,n)
• Useful in edge extraction and image sharpening
– Band-pass filter• hBP(m,n) = hL2(m,n) – hL1(m,n)
• Useful in edge enhancement• Also good for high-pass tasks in the presence of noise
– avoid amplifying high-frequency noise
ELE 488 F06
Example: Image Sharpening
B(m,n) = A(m,n) +a A(m,n) x M(m,n)
Original circuit image is from Matlab Image Toolbox.
0 -¼
-¼ 1
-1 0 1
-1
0
1
0
-¼
0 -¼ 0
ELE 488 F06
Image Sharpening: Edge Enhancement
Use LPF to generate HPF
– Subtract a low pass filtered image ILP from the original I0
– HPF extracts edges and transitions
Enhance edges
0 20 40 60 80 1000
10
20
30
40
50
0 20 40 60 80 1000
10
20
30
40
50
0 20 40 60 80 100-10
-5
0
5
10
0 20 40 60 80 1000
10
20
30
40
50
Edge
I0
IHP
ILP
IS
ELE 488 F06
Example of Image SharpeningOriginal moon image is from Matlab Image Toolbox.UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
ELE 488 F06
Gradient, 1st-order Derivatives, and Edges
• Edge: where luminance changes abruptly• For binary image
– Take black pixels with immediate white neighbors as edge pixel
• For continuous-tone image– Find luminance gradient
• partial derivatives along two orthogonal directions
• gives the direction with highest rate of changes
• If gradient is larger than a threshold => edge
– To represent edge• Edge map ~ edge intensity & directions
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
The Gradient of Function f(x,y)
The gradient of a smooth function f(x,y) at (x,y) is
derivative of f in the x-direction, partial derivative of f w.r.t. x
derivative of f in the y-direction, partial derivative of f w.r.t. y
2-vector pointing in the direction of greatest increase of f fr
.
rate of change of f in the direction h f
ELE 488 F06
Example
ELE 488 F06
Gradient of Example Function
Gradient is a vector field:a 2-vector associated with each point (x,y)
ELE 488 F06
Gradient of Discrete Function (Symmetric Difference)
]j[nΔ
fΔ
The symmetric difference of a real-valued discrete function f [n] at the point j is
The symmetric rate of change of f around the point j
)1]f[j1][jf(2
1]j[
nΔ
fΔ
ELE 488 F06
The Discrete Gradient
The gradient of a real-valued discrete function f(m,n) at (j.k) is
The symmetric rate of change of f in the m-direction at (j,k)
The symmetric rate of change of f in the n-direction at (j,k)
(Note: for simplicity, drop the scale factor of ½ - it can always be reinserted again if needed.)
ELE 488 F06
The Gradient Vector Field (GVF)
The gradient at pixel is the 2 vector
The gradient assigns a 2-vector at each pixel. This is called the gradient vector field.
Illustration of the gradient vector field showing a scaled version of the gradient at every 3rd pixel. The red lines illustrate the direction and magnitude of the gradient vector at each pixel.
Note: the gradient points in the direction of increasing image values.
ELE 488 F06
The Discrete Gradient
Filtering with the mask[-1 0 1] yields:
Filtering with the mask[-1 0 1]T yields:
The gradient can be computed by applying two LSI filters
mΔ
fΔ
nΔ
fΔ
ELE 488 F06
Averaging GVF
• gradient is computed using high pass filters (discrete differentiation) will amplify noise.
• So average neighboring gradient vectors using a spatial filtering on each of two gradient components.
• The averaging filters and gradient filters are LSI and hence can be applied in any order, i.e.,
• Can compute gradient and average, or• Average pixels and then compute gradient.
Should we average uniformly in all directions or along a selected direction? e.g. along the perpendicular to the direction of the derivative? Perpendicular to the gradient?
ELE 488 F06
Averaging the GVF
Spatial averaging filter perpendicular to direction of discrete derivative
discrete derivative filter in horizontal direction
Combination of the two masks gives single “averaged gradient mask” in horizontal direction.
ELE 488 F06
Common GVF Masks
– Apply both masks to image and combine results to determine magnitude and angle (if desired).
– Note: Prewitt and Sobel operators spatially average the horizontal and vertical differences of 3 local pixels to reduce the effect of noise
0 1
-1 0
-1 0
-1 0
1
1
-1 0 1
-1 0
-2 0
1
2
-1 0 1
1 0
0 -1
-1 -1
0 0
-1
0
1 1 1
-1 -2
0 0
-1
0
1 2 1
Roberts Prewitt Sobel
ELE 488 F06
Averaged GVF and Edges
Scaled & averaged gradient of apple image plotted at every 3rd pixel.
Large gradient magnitude
Edge:Normal to direction of gradient
Small gradient magnitude
No edge
Subjective Issue: How to select the threshold on the magnitude of the gradient to “detect” an edge?
ELE 488 F06
Edge Detection: Summary
• Edge: pixel locations of abrupt intensity change• For binary image – black pixels with immediate white
neighbors • For continuous-tone image, measure gradient vector
– Along two orthogonal directions ~ usually horizontal and verticalgx = L / x, gy= L / y
– Magnitude of gradient vectorg(m,n) 2 = gx(m,n) 2 + gy(m,n) 2
OR g(m,n) = |gx(m,n) | + |gy(m,n)| (preferred in hardware implement.)
– Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]
• Characterizing edges in an image– (binary) Edge map: specify “edge point” locations with g(m,n) > thresh.– Edge intensity map: specify gradient magnitude at each pixel– Edge direction map: specify directions
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
Example: Edge Detection from GVF
Threshold magnitude of the GVF:Above threshold: edge pixelBelow threshold: no edge
ELE 488 F06
Sobel GVF
Sobel
Prewitt
ELE 488 F06
Example: Edge Detectors Based on the GVF
Roberts Prewitt Sobel
Threshold the magnitude of the GVF:Above threshold – pixel is on an “edge”Below threshold – no “edge”
ELE 488 F06
Examples of
Edge Detectors
– Quantize edge intensity to 0/1:• set a threshold• white pixel denotes
strong edge
Roberts Prewitt Sobel
UMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)
ELE 488 F06
Example Edges in a Grayscale Image
Line 1 Line 2
Line 3 Line 4
Line 1
Line 2
Line 3
Line 4
Close up
ELE 488 F06
Common Gradient Operators for Edge Detection
– Move the operators across the image and take the inner products• Magnitude of gradient vector g(m,n) 2 = gx(m,n) 2 + gy(m,n) 2
• Direction of gradient vector tan –1 [ gy(m,n) / gx(m,n) ]
– Gradient operator is HPF in nature ~ could amplify noise• Prewitt and Sobel operators compute horizontal and vertical differences of
local sum to reduce the effect of noise
0 1
-1 0
-1 0
-1 0
1
1
-1 0 1
-1 0
-2 0
1
2
-1 0 1
1 0
0 -1
-1 -1
0 0
-1
0
1 1 1
-1 -2
0 0
-1
0
1 2 1
Roberts
H1(m,n)
H2(m,n)
Prewitt Sobel
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
Suppressing Noise via Spatial Averaging
• Image with iid noise y(m,n) = x(m,n) + N(m,n)• Averaged version
v(m,n) = (1/Nw) x(m-k, n-l) + (1/Nw) N(m-k, n-l)
• Noise variance reduced by a factor of Nw
– Nw ~ # of pixels in the averaging window
• SNR improved by a factor of Nw if x(m,n) is constant in local window
• Window size is limited to avoid blurringUM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
Coping with Salt-and-Pepper NoiseMatlab Image Toolbox Guide Fig.10-12, 10-13
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
Median Filtering
• Salt-and-Pepper noise– Isolated white/black pixels spread randomly over the image
– Spatial averaging filter may incur blurred output
• Median filtering– Take median value over a small window as output ~ nonlinear
• Median{ x(m) + y(m) } Median{x(m)} + Median{y(m)}
– Odd window size is commonly used• 3x3, 5x5, 7x7
• 5-pixel “+”-shaped window
– Even-sized window ~ take the average of two middle values as output
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
& R
.Liu
© 2
00
2)
ELE 488 F06
Other Variations of Image Sharpening
• High boost filter (Gonzalez-Woods 2/e pp132 & pp188)
I0 ILP IHP = I0 – ILP I1 = (b-1) I0 + IHP
– Equiv. to high pass filtering for b=1
– Amplify or suppress original image pixel values when b2
• Combine sharpening with histogram equalization
UM
CP
EN
EE
40
8G
Slid
es
(cre
ate
d b
y M
.Wu
© 2
00
2)
Top Related