Time-Frequency Analysis and Filtering based on the Short-Time Fourier Transform
Texture Filtering Fourier Transform and
Transcript of Texture Filtering Fourier Transform and
![Page 1: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/1.jpg)
Center for Image Processing
Fourier Transform andTexture Filtering
Lucas J. van Vlietwww.ph.tn.tudelft.nl/~lucas
![Page 2: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/2.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 2
Center for Image Processing
Image Analysis Paradigm
Texture filtering
analysis
segmentation
Imagerestoration
Imageenhancement
sensorImageformation scene
pre-processing
classification
Texture properties:scale
orientationLocal variance
regularity
![Page 3: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/3.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 3
Center for Image Processing
Texture properties
� Texture is domain property rather than a pixel property.
� Fourier analysis offers a powerful tool to analyze and describe domain properties
� Properties� Local power = local variance� Scale (from fine detail to course structures)� Anisotropy (orientation preference)
![Page 4: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/4.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 4
Center for Image Processing
Discrete Fourier Transform
� Each image can be decomposed in weighted sum of complex exponentials (sines and cosines) of frequency fand angle φ. (or two frequency components u and v)
( ) ( )( )
( ) ( )( )
1 1 2
0 0
1 1 2
0 0
1, ,
1, ,
N N j ux vyN
u v
N N j ux vyN
u v
g x y G u v eN
G u v g x y eN
π
π
− −+
= =
− −− +
= =
=
=
∑∑
∑∑
� For real-valued images: ( ){ } ( ){ }
( ){ } ( ){ }
Ev , Re ,
Od , Im ,
F
F
g x y G u v
g x y j G u v
←→
←→
image sizeNxN
![Page 5: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/5.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 5
Center for Image Processing
Getting used to Fourier (2)
An image is a weighted sum of cos (even) and sin (odd) images.
a+jb
Fourier domain with complex amplitude: a+jb
![Page 6: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/6.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 6
Center for Image Processing
Getting used to Fourier (1)
� Graphite surface by Scanning Tunneling Microscopy� Atomic structure of graphite shows a hexagonal surface
column
row
0
0N-1
N-1
(c,r) = (½N,½N)(u,v) = (0,0)
x
y
u
v
(c,r) = (0,0)(u,v) = (-½N,-½N)
(c,r) = (N-1,N-1)(u,v) = (½N-1, ½N-1)
![Page 7: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/7.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 7
Center for Image Processing
Superposition� Fourier spectrum
= + + +
= + + +
F F F F F
![Page 8: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/8.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 8
Center for Image Processing
Two domains
� The coordinate systems (x,y) and (u,v) have their origin at the position (column,row) = (½N, ½N)
u
v
x
y
column
row
column
row
0
0N-1
N-1
(u,v) = (0,0)
![Page 9: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/9.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 9
Center for Image Processing
Local variance filter� Recipe: local variance filter(filter size = n)
� Compute the local mean (blurring filter of size n)� Subtract (if necessary) the local mean.� Compute the square of each pixel value.� Suppress the “double” response by local averaging (blurring filter
of size n).� Local variance is a measure for the local squared-contrast.
![Page 10: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/10.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 10
Center for Image Processing
Segmenting textured objects� Simple thresholding fails!� The objects differ in texture from the background� Local variance followed by a (morphological) closing yields …
![Page 11: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/11.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 11
Center for Image Processing
Scaling: local vs global
� Problem: Choosing the proper scale is an important, but tedious task.� Scale too small: Texture characteristics are missed
which yields an incomplete data description.� Scale too large: Confusion (mixing) of adjacent texture
characteristics, lack of localization, and blindness for small and subtle textured regions.
� Solution: Multi-scale analysis.� Analyze the image as function of scale: from fine detail
to course “image-filling” objects.
( ) ( )( ) ( )
2 22 2 22 2
22 22
12
, ,u vx y N N
Fg x y e G u v eπ π
σ
σ
πσ
− + + −
= ←→ =
![Page 12: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/12.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 12
Center for Image Processing
Multi-Scale
Series of images filtered of decreasing scale: Scale-space� Sample the scales logarithmically using filters of size = basescale
yields n scales per octave
Input imagescale 0
scale 1
scale 2
scale 3
scale 4
scale 5
var (scale 1)
var (scale 2)
var (scale 3)
var (scale 4)
var (scale 5)
Local variance between scales n and n-1.
{ }11 13212 ,2 ,2 ,...,2 nbase ∈
Scale difference ≈ scale derivative
Scal
e sp
ace
Scal
e sp
ace
![Page 13: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/13.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 13
Center for Image Processing
Scale-spaces
� Morphological scale-space: Use openings (closings)� Gaussian scale-space: Use Gaussian filters
Increasing scales
Fourier domain with “footprints” of Gaussianfilters of increasing scale
Filter size is inversely proportional to “footprint” in Fourier domain
12
345
![Page 14: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/14.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 14
Center for Image Processing
Chirp exampleSc
ale-
spac
e
Scale derivative Spatial variance
Scal
e-sp
ace
![Page 15: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/15.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 15
Center for Image Processing
Gabor: scale and orientation
� Problem: Crossing and touching of oriented patterns.� Isotropic scale cannot discriminate based on orientation.
� Solution: Multi-orientation analysis� Orientation selective filtering
� Multi-scale and multi-orientation analysis by Gabor filtering.� Orientation selectivity
( ) ( )2 2
2 222
1, 2
,x y
i iNi i
j u x v yu vg x y e e
π
σ
πσ
+−
+=
Gaussian window of size σSelected frequency (ui,vi)
![Page 16: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/16.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 16
Center for Image Processing
Gabor filter bank
� Linear sampling of orientation� Logarithmic sampling of scale� (Remember: reciprocal relation of size in space and freq.
A single input imageyields many output images.
Question: Which output image(s) discriminate between textures?
( )
( )( ) ( )( )
( )
2 2 22 2
2, ,,
u u v vi iN N
i iu v
i i
G u v eG u u v v
π πσ
− − + −
=
= − −(ui,vi) σ−1
![Page 17: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/17.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 17
Center for Image Processing
Gabor filtering
� Each Gabor filter selects a (Gaussian-shaped) region of the Fourier spectrum of the image.
� A Gabor filter can be implemented by convolution!
( ) ( )( ) ( )( )[ ]2 2
222
2 2 1, 2
, cos sini i i ii i N N
x yu x v y u x v yu vg x y j eπ π σ
πσ
+−
+ += +
![Page 18: Texture Filtering Fourier Transform and](https://reader030.fdocuments.in/reader030/viewer/2022020704/61fb62742e268c58cd5d88f8/html5/thumbnails/18.jpg)
Center for Image ProcessingCenter for Image Processing
CBP course: Fourier & Texture 18
Center for Image Processing
How to go further …?
� Problem: How to segment� textured objects from background� adjacent textures
Each pixel is described by many texture attributes: a feature vectorPixels from a single texture yield a cluster in feature space.
� Solutions:� Unsupervised: Cluster analysis� Supervised: Pattern recognition
allows training a classifier based on examples