CBIR: Texture Features - 2...CBIR: Texture Features - 2 COMPSCI.708.S1.C A/P Georgy Gimel’farb....
Transcript of CBIR: Texture Features - 2...CBIR: Texture Features - 2 COMPSCI.708.S1.C A/P Georgy Gimel’farb....
CBIR: Texture Features - 2
COMPSCI.708.S1.C
A/P Georgy Gimel’farb
Semester 1, 2006 Lecture G8 2
Gabor Features
• Multi-resolution texture representation basedon Gabor filters– Image representation using Gabor filter responses
minimises the joint space– frequency uncertainty
– The filters are orientation- and scale-tunable edgeand line detectors
– Statistics of these local features in a region relate tothe underlying texture information
Semester 1, 2006 Lecture G8 3
Gabor Filters / Wavelets
• 1D Gabor function and its Fourier transform:
• Product of a Gaussian and a complex-plane wave
– Minimal joint uncertainty in both spatial and frequency domain
!
"(x | x0,# x,$x ) = 1
2%# x
exp & 12
(x&x0 )2
# x2( ) + 2%j$x x[ ]
'(u |# u,$x ) = exp & 12
(u&$ x )2
# u2( )[ ] where # u = 1
2%# x
Semester 1, 2006 Lecture G8 4
Gabor Filters / Wavelets
• Signal s is encoded by its projection onto these functions• Decomposition is equivalent to a Fourier transform of the
signal s (x) in a Gaussian window:
where * denotes the complex conjugate• The filters form a complete but non-orthogonal basis for
expanding a signal and gertting its localised description interms of spatial frequencies
!
S(x |"x,#
x) = s($ u)%&(x ' u |"
x,#
x)du
Semester 1, 2006 Lecture G8 5
Gabor Filters / Wavelets
Wave: an oscillating function overinfinite temporal or spatial domainhaving infinite energy E.g. a pure sinusoid sin(2πωx)
Wavelet: a small wavewith localised in time orspace finite energy
real imaginary
Semester 1, 2006 Lecture G8 6
Gabor Filters / Wavelets
• 2D Gabor function: γ(x,y) = γ(x|x0,σx,ξx)·γ(y|y0,sy,ξy)• Self-similar Gabor wavelets
– By dilating and rotating the mother function g(x,y)• KM wavelets with K orientations θn and M scales m:
!
"mn (x,y) = a#m"
x cos$n + y sin$nam
,#x sin$n + y cos$n
am
%
& '
(
) * ;
$n=n+K
; a > 1; integer m,[1,M], n,[0,K#1]
Semester 1, 2006 Lecture G8 7
Gabor Filters / Wavelets
2D odd wavelets 2D even wavelets
Semester 1, 2006 Lecture G8 8
Gabor Filter Dictionary
• Gabor wavelets are not orthogonal ⇒ redundant data!
• Design: to choose a proper set (dictionary) of wavelets– Dictionary design ⇒ to reduce the data redundancy
• Design strategy: to ensure touching each other half-peak magnitude supports of the filter responses in thespatial frequency spectrum– Filter parameters σx, σy ⇒ from the lower and upper central
frequencies ξx, ξy of the spectral domain and the number Mof the scales
Semester 1, 2006 Lecture G8 9
Gabor Filter Dictionary
Example: Lower frequency 0.05; upper frequency 0.40; N = 6orientation angles, and M = 4 scales 24 filters
Semester 1, 2006 Lecture G8 10
Similarity of Feature Vectors
• Minkowski distance between two vectors of size L:
– City-block (absolute): p = 1; Cartesian: p = 2
• Euclidean distance:
– Weighted distance: weight
!
D(v1,v
2) = "
l(v1,l# v
2,l)2
l=1
L
$% & ' ( ) *
12!
D(v1,v2) = | v1,l " v2,l |p
l=1
L
#$ % & ' ( )
1p
!
"l
= 1(variance #
l
2)
Semester 1, 2006 Lecture G8 11
Similarity of Feature Vectors
• Mahalanobis distance:
αkl – the component of the inverse covariance matrix A = Σ−1
• Accounts for variances of and statistical dependency betweenthe vector components vi,l; l = 1, …, L; i = 1, 2
!
D(v1,v
2) = (v
1" v
2)
TS"1(v1" v
2)[ ]
12
= #kl(v1,k" v
2,k)(v
1,l" v
2,l)
k=1
L
$l=1
L
$%
& '
(
) *
12
Semester 1, 2006 Lecture G8 12
Similarity of Distributions
• Kullback-Leibler (KL) divergence
where f. = (f.,t: t = 1,…,T) is a frequency distribution
• Chi-square (χ2) distance:
• Symmetric distance: (D(f1, f2) + D(f2, f1))/2
)log(loglog),(1
,2,1,1
1 ,2
,1
,121 !!==
"#=T
t
ttt
T
t t
t
t ffff
ffD ff
!
D(f1,f2) = f
1,t " f2,t( )
2f1,t
t=1
T
#
Semester 1, 2006 Lecture G8 13
MPEG-7 Texture Descriptors
• The MPEG-7 ISO/IEC standard for multimedia contentdescription interface involves descriptors for colour,texture, shape, and motion
• Three texture descriptors at this time:– the texture browsing descriptor ⇒ directionality, regularity
and coarseness of a texture– the homogeneous texture descriptor (HTD) ⇒ description
of homogeneous texture regions for similarity retrieval– the local edge histogram descriptor ⇒ for inhomogeneous
in texture properties underlying regions
Semester 1, 2006 Lecture G8 14
Texture Browsing Descriptor
• 12 bits (maximum) to characterise a texture’s regularity: 2bits, directionality: 3 bits × 2, and coarseness: 2 bits × 2
– The specification allows a maximum of 2 different directionsand coarseness values
– Regularity scale (2 bits): 0 (irregular, or random)…3 (periodic)
Semester 1, 2006 Lecture G8 15
Texture Browsing Descriptor
• Directionality is quantified to 6 values: 0o, 30o,…, 150o
• Bank of scale / orientation selective band-pass Gaborfilters to select up to 2 dominant directions and computethe descriptor components from the filtered outputs:– 3 bits per each direction: 0 – no dominant directionality; 1 – 6
to code this dominant direction– Coarseness (2 bits per each dominant direction): 0 – a fine
grain texture … 3 – a coarse texture
• Description relates to the partitioning of the frequencydomain by the filter bank being used also for the HTD
Semester 1, 2006 Lecture G8 16
Homogeneous Texture Descriptor
• Means and standard deviations of the outputs of the bankof Gabor filters (Gaussians in the polar coordinates):
– 30 output channels in a normalised frequency space 0 ≤ r ≤ 1:6 angular ϕj = 30oj and 5 radial divisions ri = r02−i; r0 = 0.75
– 62 8-bit numbers per image or region: signal mean / deviationand logarithmically scaled energy / deviation in each channel
!!"
#
$$%
& ''
!!"
#
$$%
& ''=
2
,
2
2
,
2
,2
)(exp
2
)(exp),(
j
j
ir
ijiji
ss
rrrG
(
(((
Semester 1, 2006 Lecture G8 17
Homogeneous Texture Descriptor
ωx
ωy
ϕ
Semester 1, 2006 Lecture G8 18
Edge Histogram Descriptor
• 2x2 edge detector to each image block by averaging signals
• Edge detectors:
• Histogram: 5 bins×16 subimages= 80 bins; 3 bits/bin; an edge blockif the maximum of the edge strengths exceed a preset threshold
16 subimages
Fixed numberof blocks ineach subimage
Each image block⇒ into 2x2 blocksof pixels
1 −11 −1
1 1−1 −1
√2 0 0 −√2
2 −2−2 2
0 √2−√2 0