Lecture 3 Intensity Transformations and Spatial Filtering 03...Lin ZHANG, SSE, 2016 Lecture 3...
Transcript of Lecture 3 Intensity Transformations and Spatial Filtering 03...Lin ZHANG, SSE, 2016 Lecture 3...
Lin ZHANG, SSE, 2016
Lecture 3Intensity Transformations
and Spatial Filtering
Lin ZHANG, PhDSchool of Software Engineering
Tongji UniversityFall 2016
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
About Image Enhancement
• Image enhancement is the process of making images more useful
• The reasons for doing this include:• Highlighting interesting detail in images• Removing noise from images• Making images more visually appealing
Lin ZHANG, SSE, 2016
About Image Enhancement—Several Examples
original image result of image enhancement
Lin ZHANG, SSE, 2016
About Image Enhancement—Several Examples
original image result of image enhancement
Lin ZHANG, SSE, 2016
About Image Enhancement—Several Examples
original image result of image enhancement
Lin ZHANG, SSE, 2016
About Image Enhancement
• Spatial domain VS frequency domain• Spatial domain techniques directly manipulate image pixels
• Frequency domain techniques manipulate Fourier transform or wavelet transform of an image
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions
• Image negatives• Thresholding• Log transformations• Power‐law (Gamma) transformations• Piecewise‐linear transformation functions
• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
Image Negatives
• Most spatial domain enhancement operations can be reduced to the form
where f(x, y) is the input image, g(x, y) is the processed image and T is some operator defined over some neighborhood of (x, y)
( , ) ( , )g x y T f x y
Lin ZHANG, SSE, 2016
Image Negatives
• Most spatial domain enhancement operations can be reduced to the form
( , ) ( , )g x y T f x yOrigin x
y Image f (x, y)
(x, y)
Lin ZHANG, SSE, 2016
Image Negatives
• When neighborhood is simply the pixel itself ( , ) ( , )g x y T f x y
( )s T rwhere s refers to the processed image pixel value and rrefers to the original image pixel value. In this case T is referred to as a grey level transformation function or a point processing operation
Lin ZHANG, SSE, 2016
Image Negatives
• The negative of an image with intensity levels [0, L‐1] is
• Used for enhancing white or gray details embedded in dark regions, especially when the black areas are dominant in size
( ) 1s T r L r
An X‐ray image of two hands (a) Negative image of (a)
Lin ZHANG, SSE, 2016
Image Negatives
• The negative of an image with intensity levels [0, L‐1] is
• Used for enhancing white or gray details embedded in dark regions, especially when the black areas are dominant in size
( ) 1s T r L r
Implementation Tips
1. You can write your own code since it is straightforward2. Or you can use the routine imcomplement3. Or you can use the routine imadjust
Lin ZHANG, SSE, 2016
Thresholding
• Thresholding transformations are particularly useful for segmentation in which we want to isolate an object of interest from a background
1,0,
r thresholds
otherwise
An X‐ray image of two hands (a) Thresholding result of (a)
Lin ZHANG, SSE, 2016
Thresholding
• Thresholding transformations are particularly useful for segmentation in which we want to isolate an object of interest from a background
Implementation Tips
1. You can write your own code since it is straightforward2. Or you can use the routine im2bw
1,0,
r thresholds
otherwise
Lin ZHANG, SSE, 2016
Log Transformations
• The general form of the log transformation is s = c * log(1 + r), where c is a constant, and r>=0
• It can expand the values of dark pixels in an image while compressing the higher level values
• It compresses the dynamic range of images with large variations in pixel values (such as Fourier spectrum)
(a) original image; (b) Fourier magnitude of (a) in linear scale; (c) Fourier magnitude of (a) shown in log scale
(a) (b) (c)
Lin ZHANG, SSE, 2016
Power‐Law (Gamma) Transformations
• Power law transformations have the following form
• Power‐law curves with fractional values of map a narrow range of dark input values to a wider range of output values, with the opposite being true for higher values of input levels. Varying gives a whole family of curves.
s crwhere c and are positive constants
Lin ZHANG, SSE, 2016
Power‐Law (Gamma) Transformations
• Power law transformations have the following forms cr
where c and are positive constants
Plots of for various values of . All curves were scaled to fit in the range shown.
s cr
Lin ZHANG, SSE, 2016
Power‐Law (Gamma) Transformations
• Power law transformations have the following forms cr
where c and are positive constants
Implementation Tips
1. You can write your own code since it is straightforward2. Or you can use the routine imadjust
Lin ZHANG, SSE, 2016
Power‐Law (Gamma) Transformations
• Power law transformations are useful for general‐purpose contrast manipulation
s = r 0.6
s = r 0.4
Lin ZHANG, SSE, 2016
Power‐Law (Gamma) Transformations
s = r 3.0
s = r 4.0
Lin ZHANG, SSE, 2016
Another Contrast Stretching Function
• The following function is another commonly used contrast stretching function
1
1 / Esm r
where m and E are two parameters
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
m
Lin ZHANG, SSE, 2016
Another Contrast Stretching Function—An Exampleoriginal input constrast stretched result
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• The advantage of piece‐wise linear functions is that its form can be arbitrarily complex; the disadvantage is that their specification requires more user input
• Three usually used piece‐wise linear transformations• Contrast stretching• Intensity‐level slicing• Bit‐plane slicing
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• Contrast stretching is a process that expands the range of intensity levels in an image so that it spans the full intensity range of the recording medium or display device
Input image Form of piece‐wise function Output image
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• Intensity level slicing highlights a specific range of intensities in an image
(a) This transformation highlights intensity range [A, B] and reduces all the other intensities to a lower level
(b) This transformation highlights intensity range [A, B] and preserves all the other intensities
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• Intensity level slicing highlights a specific range of intensities in an image
(a) Aortic angiogram; (b) result of using a slicing transformation of the type (a) (previous slide); (c) result of using a slicing transformation of the type (b) (previous slide);
(a) (b) (c)
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• Bit‐plane slicing isolates particular bits of the pixel values and can highlight interesting aspects of that image• Higher‐order bits usually contain most of the significant visual information
• Lower‐order bits contain subtle details
Lin ZHANG, SSE, 2016
Bit‐plane slicing—an example
Image of our school
Lin ZHANG, SSE, 2016
Bit‐plane slicing—an example
bp1bp0
bp2 bp3 bp4
bp5 bp6 bp7
Lin ZHANG, SSE, 2016
Bit‐plane slicing—an example
Original image reconstruction by highest 3 bitplanes
reconstruction by highest 4 bitplanes reconstruction by highest 5 bitplanes
Lin ZHANG, SSE, 2016
Piece‐wise Linear Transformations
• Bit‐plane slicing• Decomposing an image into its bitplanes is useful for analyzing the relative importance of each bit in the image, a process that aids in determining the adequacy of the number of bits used to quantize the image
• Such a decomposition is also useful for compression
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions• Histogram processing
• A taste of probability• Introduction of histogram• Histogram computation• Properties of the histogram• Histogram equalization• Using histogram statistics for image enhancement
• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
I and my histogram!
Lin ZHANG, SSE, 2016
If is continuous at x, we have
A Taste of Probability
• Probability density function (PDF)• PDF f(x) of a continuous random variable, is a function that describes the relative likelihood for this random variable to take on a given value
( ) 0, ( ) 1f x f t dt
It satisfies the following requirements,
(1)
2
11 2 2 1( ) ( ) ( ) ( )
x
xP x X x F x F x f t dt (2)
(3) ( )f x ' ( ) ( )F x f xwhere F(x) is the cumulative distribution function of X
Lin ZHANG, SSE, 2016
A Taste of Probability
• Cumulative distribution function (CDF)• F(x) describes the probability that a real‐valued random variable X with a given probability distribution will be found at a value less than or equal to x.
( ) ( ) ( )x
P X x F x f t dt
( ) ( ) ( ) ( )
b
aP a X b F b F a f t dt
Lin ZHANG, SSE, 2016
A Taste of Probability
• PDF of the uniform distribution1 ,
( )0,
a x bf x b a
otherwise
Corresponding CDF?
Lin ZHANG, SSE, 2016
A Taste of Probability
• PDF of the Gaussian (Normal) distribution2
2( )
21( ) ,2
x
f x e x
where and are constants, and 0
Johann Carl Friedrich Gauss (30 April 1777 – 23 February 1855) was a German mathematician and physical scientist who contributed significantly to many fields
Lin ZHANG, SSE, 2016
A Taste of Probability
• PDF of the Gaussian (Normal) distribution2
2( )
21( ) ,2
x
f x e x
where and are constants, and 0
Lin ZHANG, SSE, 2016
A Taste of Probability
• Expectation of the random variable X with f(x) as its probability density function
( ) ( )E X xf x dx
It can be regarded as a weighted mean or average
Theorem If the random variable Y is the function of the random variable X, Y=g(X), its expectation is
( ) ( ( )) ( ) ( )E Y E g X g x f x dx
Lin ZHANG, SSE, 2016
A Taste of Probability
• Variance• The variance is a measure of how far a set of numbers is spread out
2
2
( ) ( )
( ( )) ( )
D X E X E X
x E X f x dx
Variance of a random variable X is defined as
( )D X is called the standard deviation of X
Lin ZHANG, SSE, 2016
A Taste of Probability
• Variance
22( ) ( ) ( )D X E X E X Theorem
Proof:
2
22
22
22
( ) [ ( )]
2 ( ) ( )
( ) 2 ( ) ( ) ( )
( ) ( )
D X E X E X
E X XE X E X
E X E X E X E X
E X E X
Lin ZHANG, SSE, 2016
A Taste of Probability
• Expectation and variance of Gaussian distribution
If , that means 2~ ( , )X N 2
2( )
21( )2
x
f x e
Then,
2
( )( )
E XD X
How to prove it? Assignment
Lin ZHANG, SSE, 2016
A Taste of Probability
• Covariance• covariance is a measure of how much two random variables change together
cov( , ) ( ) ( )X Y E X E X Y E Y X and Y are two random variables, their covariance is defined as
Correlation coefficient
cov( , )( ) ( )XY
X YD X D Y
Lin ZHANG, SSE, 2016
Introduction of Histogram
• The histogram h of an image represents the frequency with which different grey‐levels occur
• Suppose f is an image represented by a total of b bits, in which case . Then h(x) is equal to the number of pixels in f, whose value is x
: [0,2 1]bf
Lin ZHANG, SSE, 2016
Introduction of HistogramExamplef is a 3‐bit 44 image
0 31 2 4 5 6 7intensity levels
pixel occurrence5 5 3 3
33340 61
77 2 2
2
Lin ZHANG, SSE, 2016
Introduction of HistogramExample
0 31 2 4 5 6 7intensity levels
pixel occurrence
1 1
3
5
1 122
2 1
0
( )b
x
h x area of the image
f is a 3‐bit 44 image
5 5 3 33334
0 61
77 2 2
2
Lin ZHANG, SSE, 2016
Histogram Computation
Algorithm: hist_comp (im) //im is a b-bit image1. Assign zeros to all elements of the array
[ ] : [0,2 1]bh i i
2. Loop all the pixels (x, y) of the image im, and
[ ( , )] [ ( , )] 1h im x y h im x y
Implementation Tips
1. You can use the routine imhist
Lin ZHANG, SSE, 2016
Properties of the Histogram
• h(x) does not depend on the locations of image pixels. Thus, it is impossible to reproduce f from its histogram.
• Usually the histogram is normalized by the area of the image (number of pixels). This gives the histogram an interpretation of a (discrete) probability density function (PDF) with properties:
1
0
0 , ( ) 0, ( ) 1L
x
x L h x h x
where L is the number of possible intensity levels
Lin ZHANG, SSE, 2016
Contrast difference?
Properties of the Histogram
Lin ZHANG, SSE, 2016
Histogram Equalization
• Motivation• An image whose pixels tend to occupy the entire range of possible intensity levels and tend to be distributed uniformly, will have an appearance of high contrast
Lin ZHANG, SSE, 2016
Histogram Equalization
• Motivation
f T(f)Intensity transformation T
0 L‐1intensity levels
occurrence probability
0 L‐1intensity levels
occurrence probability
11L
T
• T can make f have a uniform histogram• Applying T to f is called the histogram equalization of f
How to find such a T?
Lin ZHANG, SSE, 2016
Histogram EqualizationLet’s consider the continuous case first for simplicity
Take the intensity values of f as a random variable X
Its normalized histogram is the PDF of X( )Xh x
Such a T:
0( ) ( 1) ( )
X
XY T X L h t dt can satisfy the requirements How to
verify?
Lin ZHANG, SSE, 2016
Histogram Equalization( )Y T X
0
1( ) ( ) ( )( 1) ( )
1 1( )( 1) ( ) 1
Y X X x
X
XX
dxh y h x h xdy d L h t dt
dx
h xL h x L
That means after applying T to f, the resulting image has a uniform histogram
Lin ZHANG, SSE, 2016
Histogram Equalization
Let’s consider the discrete case
For a b‐bit image with size m n, the corresponding histogram equalization transform T maps intensity p to
0 0
2 1( ) (2 1)bp p
b kk
k k
nT p nm n m n
where denotes the number of pixels with intensities kkn
Lin ZHANG, SSE, 2016
Histogram Equalization
• Notes on discrete case for HE• Histogram is an approximation to a PDF, thus, perfectly flat histograms are rare in practical applications
• HE tends to spread the histogram of the input image to a wider range of intensity scale. The net result is contrast enhancement
• HE is a fully automatic procedure; no parameters
Lin ZHANG, SSE, 2016
Histogram Equalization—Examples
0
500
1000
1500
2000
2500
3000
3500
4000
0 50 100 150 200 250
0
500
1000
1500
2000
2500
3000
3500
4000
0 50 100 150 200 250
Lin ZHANG, SSE, 2016
Histogram Equalization—Examples
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 50 100 150 200 250
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
0 50 100 150 200 250
Lin ZHANG, SSE, 2016
Histogram Equalization—Examples
0
1000
2000
3000
4000
5000
6000
7000
8000
0 50 100 150 200 250
0
1000
2000
3000
4000
5000
6000
7000
8000
0 50 100 150 200 250
Lin ZHANG, SSE, 2016
Summary about Histogram
• Histogram represents the intensity occurrences of a given image
• Histogram is a global descriptor; reflect no structural information; two totally different images can have identical histograms
• Histogram has many potential applications in DIP• Histogram equalization can enhance the image’s contrast
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• Linear VS nonlinear operations
( , ) ( , )H f x y g x y
Consider a general operator H, that produces an output image g(x, y), for a given input image f(x, y):
H is said to be a linear operator if
( , ) ( , )
( , ) ( , )
( , ) ( , )
i i j j
i i j j
i i j j
H a f x y a f x y
a H f x y a H f x y
a g x y a g x y
Otherwise, H is said to be a nonlinear operator
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• Linear VS nonlinear operations• Filters can be categorized as linear and nonlinear ones based on the properties of the associated operations
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• The mechanics of linear spatial filtering• The filtering result at a specific position is the sum of the products between the filter kernel and the image patch
• Traverse the whole image to get the final result
Given an image f(x, y)
Form an image g(x, y)
g(x, y) is a weighted average of a patch near (x, y) in f . And the weight pattern is called as filter kernel h
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• The mechanics of linear spatial filtering
123 127 128 119 115 130
140 145 148 153 167 172
133 154 183 192 194 191
194 199 207 210 198 195
164 170 175 162 173 151
Original Image x
y
Enhanced Image x
y
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• The mechanics of linear spatial filteringr s t
u v w
x y z
Origin x
y Image f (x, y)
eprocessed = v*e + r*a + s*b + t*c + u*d + w*f + x*g + y*j + z*i
Filter hSimple 3*3
Neighbourhoode 3*3 Filter
a b c
d e f
g j iOriginal Image
Pixels
*
The above is repeated for every pixel in the original image to generate the filtered image
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• The mechanics of linear spatial filtering
( , ) ( , ) ( , )a b
s a t b
g x y h s t f x s y t
Filtering can be given in equation form
where a and b are the half‐width and half‐height
It is referred as correlation. (Moving a filter mask over the image and computing the sum of products at each location)
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10 20
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10 20 30
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10 20 30 30
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10 20 30 30 30
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f
0 10 20 30 30 30 20
1 1 1
1 1 1
1 1 1
h =
g
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
f1 1 1
1 1 1
1 1 1
h =
g
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
Mean filter
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering—An Example
Filtered by
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
h =
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• Correlation VS Convolution• Convolution is the same as correlation, except that the filter (or the signal) is first rotated 180 degree (for symmetric filters, they are the same)
• Convolution is an important concept in linear systems (we will revisit it later)
( , ) ( , ) ( , )a b
s a t b
g x y h s t f x s y t
Correlation of h and f
( , ) ( , ) ( , )a b
s a t b
g x y h s t f x s y t
Convolution of h and f
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• A vivid example for convolution
0 0 0 0 00 0 0 0 00 0 1 0 00 0 0 0 00 0 0 0 0
Result ?
Do convolution
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• A vivid example for convolution
0 0 0 0 00 0 0 0 00 0 1 0 00 0 0 0 00 0 0 0 0
Identical copyDo convolution
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• A vivid example for convolution
0 0 0 0 00 0 0 0 00 0 0 0 10 0 0 0 00 0 0 0 0
Result ?
Do convolution
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
• A vivid example for convolution
0 0 0 0 00 0 0 0 00 0 0 0 10 0 0 0 00 0 0 0 0
Do convolutionShift right by 2 pixel
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
Implementation Tips for Spatial Filtering
The following Matlab routines are commonly used, fspecial, imfilter, conv2, colfilt, medfilt2, and ordfilt2
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
From mathematical view, convolution has more good properties than correlation in theory. Thus, in most cases, for theoretical analysis, linear filtering is represented as convolution
For 1D continuous case, convolution is defined as
( )* ( ) ( ) ( )f t h t f t h d
Lin ZHANG, SSE, 2016
Fundaments of Spatial Filtering
Convolution has the following properties
1. 1 2 2 1( ) * ( ) ( ) * ( )f t f t f t f t
2. 1 2 3 1 2 1 3( ) * ( ) ( ) ( ) * ( ) ( ) * ( )f t f t f t f t f t f t f t
1 2 3 1 2 3( ) * ( ) * ( ) ( ) * ( ) * ( )f t f t f t f t f t f t3.
We will encounter it again in lecture 4 for filtering in the frequency domain
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• The output of a linear smoothing spatial filter is simply the (weighted) average of the pixels in the neighborhood defined by the filter mask; sharp transitions are reduced
• Linear smoothing filters are low‐pass filters; they will blur edges
• They can reduce irrelevant details in an image and get a gross representation of objects of interest
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Two simple square average filters
1 1 1
1 1 1
1 1 1
19
116
1 2 1
2 4 2
1 2 1
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
Images smoothed with square averaging filter masks of different sizes
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
Original Image Smoothed Image Thresholded Image
• Another example for averaging filter• By smoothing the original image we get rid of lots of the finer detail which leaves only the gross features for thresholding
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter• A linear smoothing spatial filter, whose coefficients are determined by a Gaussian function
• It assigns more weight to the positions near the center, and less weight to the positions far away from the center
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter2 2
2 2
1( , ) exp2 2
x yG x y
0.003 0.013 0.022 0.013 0.0030.013 0.059 0.097 0.059 0.0130.022 0.097 0.159 0.097 0.0220.013 0.059 0.097 0.059 0.0130.003 0.013 0.022 0.013 0.003
5 x 5, = 1
where is called standard deviation
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter• Gaussian filters have infinite support, but discrete filters use finite kernels
How to properly select the size of a Gaussian kernel for a given σ?
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter• Gaussian filters have infinite support, but discrete filters use finite kernels
• Rule of thumb: set filter half‐width to about 3σ
2~ ( , ),if X Nthen
( 3 ) 0.9974P X
3σ‐rule from the theory of probability:
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter—an example Gaussian filter used for smoothing
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter—an example
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Gaussian filter—some properties• Remove “high‐frequency” components from the image (low‐pass filter)
• Convolution with itself is another Gaussian• Separable kernel
• Factors into product of two 1D Gaussians
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Separability of the Gaussian filter• The 2D Gaussian can be expressed as the product of two functions
• In this case, the two functions are 1D Gaussian
2 2
2 2
2 2
2 2
1( , ) exp2 2
1 1exp exp2 22 2
x yG x y
x y
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters—Noise Types
• Salt and pepper noise: contains random occurrences of black and white pixels
• Impulse noise: contains random occurrences of white pixels
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Original
Gaussian noise
Salt and pepper noise
Impulse noise
Lin ZHANG, SSE, 2016
• Filtering salt‐and‐pepper noise using Gaussian filters
What’s wrong with the results?
3x3 5x5 7x7
Smoothing Spatial Filters
Alternative solution: median filters
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters
• Median filter (a nonlinear filter)• It replaces the value of a pixel by the median of the intensity values in the neighborhood of that pixel
• It can be used to remove (salt‐and‐pepper noise)
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters—Median Filter
• Median filter is robust to outliers
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters—Median Filter
• An exampleSalt-and-pepper noise Median filtered
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters—Median Filter
• Median VS Gaussian filtering
Original image Add salt&pepper noise
Lin ZHANG, SSE, 2016
Smoothing Spatial Filters—Median Filter
• Median VS Gaussian filtering
Median filter3 33 3 Gaussian filter, 0.5
Lin ZHANG, SSE, 2016
Outline
• About image enhancement• Some basic intensity transformation functions• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• They are used to highlight transitions in intensity• Remove blurring from images• Highlight edges
• Usually, such filters are based on discrete differentiation
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• First order derivative• It’s just the difference between subsequent values and measures the rate of change of the function
)()1( xfxfxf
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• Second order derivative• Simply takes into account the values both before and after the current value
2
2 ( 1) ( 1) 2 ( )f f x f x f xx
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• Laplacian operator• It is based on 2nd‐order partial derivatives• It can be used for highlighting intensity discontinuities in an image
2 22
2 2
f ffx y
The Laplacian is defined as follows:
In discrete case2 ( 1, ) ( 1, ) ( , 1) ( , 1) 4 ( , )f f x y f x y f x y f x y f x y
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• Laplacian operator
The Laplacian filter kernel is defined as
0 1 0
1 -4 1
0 1 0
When considering the diagonal dimensions, the Laplacian kernel can be
1 1 1
1 -8 1
1 1 1
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• Laplacian operator is rotation invariant, that means2 2 ' '( , ) ( , )f x y f x y
where'
'
cos sinsin cos
x xyy
Proof:
We need to verify2 2 2 2
2 2 '2 '2
f f f fx y x y
Lin ZHANG, SSE, 2016
'
'
cos sinsin cos
x xyy
From
we can have '
'
cos sinsin cos
x xy y
' '
' '
cos sinsin cos
x x yy x y
' ' ' cos ( sin )f f x f y f fx x x y x x y
Lin ZHANG, SSE, 2016
2 2 2 2 2
'2 2 ' ' ' 2 '
2 2 2 2
2 2
2 2 22
2
cos ( sin )
cos ( sin ) cos cos ( sin ) ( sin )
cos sin cos sin cos
f f x f y f x f yx x x x y x y x x y x
f f f fx x y y x y
f f fx x y y x
22
2 sinfy
In a similar way, 2 2 2 2 2
2 2'2 2 2sin sin cos sin cos cosf f f f f
y x x y y x y
Lin ZHANG, SSE, 2016
2 2 2 2
2 2 2 2'2 '2 2 2
2 2
2 2
cos sin sin cosf f f fx y x y
f fx y
Thus, we have
(completed)
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• Laplacian filter can be used to sharpen image by2( , ) ( , ) ( , )g x y f x y c f x y
where c is a constant
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• An example for Laplacian Sharpening
Input image
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• An example for Laplacian Sharpening
Laplacian image
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• An example for Laplacian Sharpening
Final result(= original image – 0.8*laplacian image)
Lin ZHANG, SSE, 2016
Sharpening Spatial Filters
• An example for Laplacian Sharpening
Input image Sharpened image
Lin ZHANG, SSE, 2016
Summary
• In this lecture, we have learned• About image enhancement• Some basic intensity transformation functions• Histogram processing• Fundamentals of spatial filtering• Smoothing spatial filters• Sharpening spatial filters
Lin ZHANG, SSE, 2016
Thanks for your attention