Exploiting Flickr Tags and Groups for Finding Landmark Photos short paper at ECIR 2009 Rabeeh...
-
Upload
vanessa-potter -
Category
Documents
-
view
216 -
download
0
Transcript of Exploiting Flickr Tags and Groups for Finding Landmark Photos short paper at ECIR 2009 Rabeeh...
Exploiting Flickr Tags and Groups for Finding Landmark
Photos
short paper at ECIR 2009
Rabeeh Abbasi, Sergey Chernov, Wolfgang Nejdl, Raluca Paiu, and Steffen Staab
{abbasi,staab}@uni-koblenz.de, {chernov,nejdl,paiu}@L3S.de
Intro: Landmarks vs Non-Landmarks Problem
Tag “Beijing”
Finding Landmark Photos
Goal Develop a method for easy classification of resources
Idea Exploit Flickr Groups (http://www.flickr.com/groups)
Method Select groups related to positive
and negative classes for training examples
Create and normalize feature space Train the classifier Classify unknown images
Applications Helps in improving search and
browsing of resources related toparticular class(es)
Query
flickr.com/photos/swamibu/2223726960/, flickr.com/photos/gunner66/2219882643/, flickr.com/photos/mromega/2346732045/, flickr.com/photos/me_haridas/399049455/, flickr.com/photos/caribb/84655830/, flickr.com/photos/conner395/1018557535/, flickr.com/photos/66164549@N00/2508246015/, flickr.com/photos/kupkup/1356487670/, flickr.com/photos/asam/432194779/, flickr.com/photos/michaelfoleyphotography/392504158/
ClassifierClassifier
Norm
alization
ClassificationModelSVMSVM
+VE -VE
PositiveTraining
Examples
NegativeTraining
Examples
Positive Flickr Groups
Negative Flickr Groups
Problem decomposition
2. Classify all photos into landmarks and non-
landmarks
1. Select all photos containing city tag
3. Rank tags from landmark photos by
likelihood to represent a landmark
4. Present top-k photos containing most
prominent landmark tags
Tag Normalization for ClassificationU - users, T - tags, R - resources (photos)Normalized tag frequency of a tag t in a resource rTag Frequency:
TFr(t) = t tag counts per photo / total tags per photoInversed Resource Frequency:
IRF(t) = log (total number of photos / number of photos having t)Inversed User Frequency:
IUF(t) = log (total number of users / number of user having t)
Feature vector for a photo r:F(r) = [TFr(t1)*IRF(t1); TFr(t2)*IRF(t2); … ; TFr(tjTj) IRF(tjTj)]
Experiments with Normalization Schema
Best schema:
F(r) = TFr(t)*IRF(t)
Measuring Tag Representativeness
City Tag Frequency, City User Tag Frequency, Confindence:
t tag counts across landmark photos for a city
CTF(t) = maximum tag counts across landmark photos for a city
number of users having tag t across landmark photos for a city
CUTF(t) = maximum number of users having tag t across landmark photos for a city
CONF(t) = log (sum of confidence scores produced by SVM from all photos with t)
RepresentativenessScore(t) = IRF(t) * IUF * CTF(t) * CUTF(t) * CONF(t)
Evaluation
Experimental Setup
Datasets - Training Dataset (430k photos) - Test Dataset (232k photos)
Comparison with state-of-art system WorldExplorer (Yahoo!)
20 Users evaluated both methods for 50 cities
Evaluation Prototype
The users were asked to judge if a photo is a landmark or not
Around 400-500 judgmentsper user (30 minutes per user).
20 users, each user evaluated two result sets (mixed together) for 10 randomly selected cities out of 50, each city is evaluated by 4 users in total
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Precision per UserWorld Explorer Our Method
User
Prec
ision
Micro-average Precision: World Explorer = 0.33 Our method = 0.37
Statistically significant
dres
den
mos
cow
ista
nbul
cairo
liver
pool
bras
iliaro
me
shan
ghai
flore
nce
prag
uetu
rinro
tterd
amco
logn
ebe
rlin
sevi
lleba
rcel
ona
geno
aam
ster
dam
glas
gow
napl
espa
lerm
oha
mbu
rgbu
char
est
hann
over
mad
ridhe
lsin
kivi
enna
beijin
gat
hens
chic
ago
birm
ingh
ambu
enos
aire
ssa
ntia
gone
wyo
rkm
unic
hto
ront
ole
eds
hong
kong
riode
jane
iroto
kyo
oslo
losa
ngel
espa
rislo
ndon
stoc
khol
myo
koha
ma
sing
apor
esa
opau
lom
exic
ocity
sydn
ey
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
Precision per City
World Explorer Our Method
Pre
cisi
on
Macro-average Precision: World Explorer = 0.33 Our method = 0.34
Not statistically significant, variance is too high
Conclusions
1. Precision improvement of 12%, (80% users preferred our method 60% cities are better than WE with our method
2. Landmark finding based on photo classification can replace geo-tagging based methods in situations where geo-spatial information is not available
The algorithm has a potential to be generalized beyond city landmarks for any topical photos, such as “cars", “mobile phones“, etc.
Thank You!