Post on 16-Jan-2015
description
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 1
Pattern Recognition and Applications GroupDepartment of Electrical and Electronic EngineeringUniversity of Cagliari, Italy
PhD Program in Electronic and Computer EngineeringPhD School in Information Engineering
Neighborhood-Based Feature
Weighting for Relevance
Feedback in Content-Based
Retrieval
Luca Pirasluca.piras@diee.unica.it
R AP
Pattern Recognition and
Applications Group
Giorgio Giacinto
giacinto@diee.unica.it
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 2
Outline
• Relevance Feedback
• Image representation
• Weighted similarity measures
• State of the art: Estimation of Feature Relevance
• Neighborhood-Based Feature Weighting
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 3
Aim of this work
• Exploiting neighborhood relations to weight
feature sets
• Weight designed to improve Relevance
Feedback based on Distance weighted kth-
Nearest Neighbor
• Dw k-NN estimate the relevance of an image
according to the (non-)relevant one in its
nearest neighborhood
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 4
Distance weighted kth-Nearest
Neighbor
relevanceNN
I( ) =p
NN
rI( )
pNN
rI( ) + p
NN
nrI( )
=I !NN
nrI( )
I !NNr
I( ) + I !NNnr
I( )
where pNN
rI( ) =
1
N
V I !NNr
I( )( )
and V I !NN I( )( )" I !NN I( )
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 5
Images
database
System
Image
Retrieval
Relevance Feedback
User
• Query by examples
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 6
Images
database
System
Image
Retrieval
Relevance Feedback
User
k best ranked images
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 7
Images
database
System
Image
Retrieval
Relevance Feedback
User
image labelling
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 8
Image representation
color textureshape
col. hist. layout color moments co-occurrence texture
I(F)
F = [ f1 … fi … fF ]
f1,1 … f1,i fFi …
f1,1,1 . . .f1,1,j . . .f1,1,32
f1,i,1 . . .f1,i,j . . .f1,i,9
fF,i,1 . . .fF,i,j . . .fF,i,16
fi,1… fi,j
level
image
feature
representation
components
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 9
Image representation
color histogram layout
f1,1 = [g1,1,1, g1,1,2, g1,1,3, g1,1,4 ]
g1,1,1 = [f1,1,1, …, f1,1,8]
g1,1,2 = [f1,1,9, …, f1,1,16]
g1,1,3 = [f1,1,17, …, f1,1,24]
g1,1,4 = [f1,1,25, …, f1,1,32]
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 10
Image similarity
S fi, j( ) = IA fi, j ,k( ) ! IB fi, j ,k( )p
k=1
N
"#$%&'(
1p
S fi( ) = S fi, j( )j
!
S = S fi( )i
! feature (higher)
representation
components (lower)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 11
Weighted similarity measures
In order to have good performance into images
retrieval systems
• Relevant images should be considered as
neighbors each others.
• Non-relevant images should not be in the
neighborhood of relevant ones.
• Weighted similarity measures.
• Weights related to the capability of featurespaces of representing relevant images as
nearest-neighbors
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 12
Weighted similarity measures
S fi, j( ) = wi, j ,k IA fi, j ,k( ) ! IB fi, j ,k( )p
k=1
N
"#$%&'(
1p
S fi, j( ) = wg idpgIA , IB( )
g=1
G
!
S fi( ) = wi, j iS fi, j( )j
!
S = wi iS fi( )i
! feature (higher)
representation
components (lower)
component subset
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 13
State of the art
• Inverse of standard deviationRui, Huang, Mehrotra. Int. Conf. on Image Processing , 1997
wfj=1
! j
• Probabilistic learning (PFRL)Peng, Bhanu, Qing. Computer Vision and Image Understanding, 1999
wfj=
eT irf j z( )( )
eT irl z( )( )
l=1
F
!
fj is the j-th feature, !j is its standard deviation
rfj(z) is the measure of relevance of the j-th
feature for the query z
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 14
Neighborhood-Based
Feature Weighting
• “Relevance” of different feature space is
estimated in terms of their capability of
representing relevant images as Nearest
Neighbors
• Relevance of an image is estimated according
to the relevant and non-relevant images in its
nearest nieghborhood
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 15
Neighborhood-Based
Feature Weighting
wfx
=p
NN
r fx( )
pNN
r fx( ) + p
NN
nr fx( )
=
dmin
fx I
i,N( )
i!R
"
dmin
fx I
i,R( )
i!R
" + dmin
fx I
i,N( )
i!R
"
where pNN
r fx( ) =
1
VNN
r fx( )
and VNN
r fx( )! 1
card(R)d
min
fx I
i,R( )
i"R
#
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 16
Neighborhood-Based
Feature Weighting
• Evaluation of capability to exploit neighborhood
relations in terms of weighted similarity measures
and in terms of weighted relevance score :
– Components level
– Component subset level
relevance
NNfi, j( )S fi, j( )
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 17
Why better?
• Inverse of standard deviation
– Doesn’t use information about neighborhood of
relevant images
• Probabilistic learning (PFRL)
– It considers only relevant images
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 18
Dataset
• Corel 19511 images
• 43 classes (min: 96 - max: 1544 images)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 19
Feature sets
• 4 feature sets
– Co-Occurrence Texture (4x4 subsets)
• 4 directions x 4 values
– Color Moments (3x3 subsets)
• first 3 moments x (H, S, V)
– Color Histogram (4x8 subsets)
• 8 ranges of H x 4 ranges of S
– Color Histogram Layout (4x8 subsets)
• 4 sub-images x 8 color
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 20
Experiment setup
• 500 queries
• 9 iterations
• 20 images retrieved each iteration
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 21
Legend
*Dw 2-NN no weight
SVM no weight
Dw 2-NN Probabilistic learning
Dw 2-NN Inverse of standard deviation
Dw 2-NN Neighborhood-Based
Dw 2-NN N-Based component subset
Dw 2-NN N-Based Score component subset
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 22
Experimental Results
Color Histogram
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 23
Experimental Results
Color Histogram
F =1
1
2 ! prec+
1
2 !recall
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 24
Experimental Results
Color Histogram (PFRL)
6-05-2009 Neighborhood-Based Feature Weighting - L. Piras 25
Conclusions
• Reported results show that a weighted measure
improve the performance of the NN technique
• Weighted distance metric based on feature
subset provided the best results
• Neighborhood-Based weights provide similar or
better results with respect to PFRL but without
annoying tuning operations