Post on 18-Jul-2020
1
Geotag Propagation in Social NetworksGeotag Propagation in Social NetworksBased on User Trust Model
Ivan Ivanov, Peter Vajda, Jong-Seok Lee,Lutz Goldmann, Touradj Ebrahimi
Multimedia Signal Processing GroupEcole Polytechnique Federale de Lausanne, Switzerland
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
2
We introduce users in our system for geotagging in order to
Motivation
We introduce users in our system for geotagging in order to simulate a real social network
GPS coordinates to derive geographical annotation, which are not available for the majority of web images and photosare not available for the majority of web images and photos
A GPS sensor in a camera provides only the location of the photographer instead of that of the captured landmark
Sometimes GPS and Wi-Fi geotaggingdetermine wrong location due to noise
htt // l t t
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.placecast.net
3
Tag short textual annotation (free form keyword) used to
Motivation
Tag – short textual annotation (free-form keyword) used to describe photo in order to provide meaningful information about it
User provided tags may sometimes be spam annotations User-provided tags may sometimes be spam annotationsgiven on purpose or wrong tags given by mistake
User can be “an algorithm”
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/scriptingnews/2229171225 http://code.google.com/p/spamcloud
4
Consider user trust information derived from users’ tagging
Goal
Consider user trust information derived from users tagging behavior for the tag propagation
Build up an automatic tag propagation system in order to:D th t ti ti d Decrease the annotation time, and
Increase the accuracy of the system
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.costadevault.com/blog/2010/03/listening-to-strangers
5Geotags
Tags• summer school• June• June• 2009• sea Geotags• Hagia Sophia
IPTC h i id d
Hagia Sophia• Blue Mosque• Istanbul• Turkey• geo:lat = 41.01667
IPTC schema is considered: City – name of the city Sublocation – area or name of the landmark
E l I t b l (H i S hi ) S F i (G ld G t B id )
g• geo:lon = 28.96667
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
Examples: Istanbul (Hagia Sophia), San Francisco (Golden Gate Bridge)
6Trust and reputation systems
Email spam filtering
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.wpconfig.com/2010/07/04/tips-to-increase-your-pagerank
7System overview
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
8
Object duplicate detection module provides a matching score
Tag propagation
Object duplicate detection module provides a matching score matrix
Two application scenarios:Cl d t bl
, 1 , 1 i , jS i ,M j ,N
Closed set problem:
11 if
0 otherwise
i , j i , ji ,Mi , j
, S max SC
Open set problem:
0 otherwise ,
11 if
0 otherwise
i , j i , j i , ji ,Mi , j
ˆ, S max S S SO
,
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
9
Three steps:
User trust modeling
Three steps: User evaluation – comparing the predicted tags to manually defined
ground truth (for a real photo sharing system, such as Panoramio, it is not necessary to collect ground truth data since user feedback can y greplace them):
1 if user tags image correctly0 otherwisei , j
, j iU
, Trust model creation – the percentage of the correctly tagged
images by particular user out of the overall number of tagged images M:
Mi jU
Tag propagation – only tags from users who are trusted ( ) are propagated to other photos in the dataset
1 i , jij
UT
Mj
ˆT T
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
10User trust modeling in Panoramio
Tagged incorrectly?
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.panoramio.com/photo/23286122
11
Idea:
User trust modeling in Panoramio
Idea:
true flag (correct tag)
false flag (wrong tag + suggest a correct tag)
Th i l t h th t t d The more misplacements a user has, the more untrusted he/she is
User trust ratio:
all images tagged by user
true flagsall associated flagsj
j
# T#
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
12
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 9 T . 0 7 T . 0 5 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
13
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 3 T . 0 7 T . 0 9 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
14
Users can collaboratively eliminate a spammer:
Eliminate spammers in Panoramio
Users can collaboratively eliminate a spammer:
Bob
Eve
Bob
Alice
EAlice
0 7 T . 0 8 T . 0 4 T .
B b
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
EveAliceBob
15Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
16Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
17Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
18Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
19Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
20Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob Eve
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
21Tag propagation based on user trust model
0 7T 0 8T 0 4T 0 6T̂ EveAlice
0 7 T . 0 8 T . 0 4 T . 0 6T .Bob
Alice
Bob Eve
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
http://www.flickr.com/photos/gustavog/9708628
22
Dataset
Experiments
Dataset 1320 images ‒ 22 cities, 3 sublocations for each city, 20 sample
images for each sublocation
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
23
Dataset
Experiments
Dataset Large variety of views, distances and partial occlusions
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
24
Methodology
Experiments
Methodology Extract a sub network from a large social network, in a way that every
user in this subsystem annotates every monument in the subset of the dataset
Upon this sub network, we build up an automatic propagation system in order to decrease the annotation time and increase the accuracy of the system44 bj t k d t t 66 h t h i d 44 subjects were asked to tag 66 photos, chosen in advance
If either one of geotags (name of the area or name of the monument depicted in the image) is correlated with the object in the image, we assume that the image is correctly taggedg y gg
Evaluation Recognition rate (accuracy):
RR TA
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
A
25
The recognition rate for all landmarks
Results
The recognition rate for all landmarks
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
26
The recognition rate across the different locations groups
Results
The recognition rate across the different locations groups (bars) and the recognition rate of all locations (dashed line)
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
27
Trust ratios for the different users for a reduced (first 20
Results
Trust ratios for the different users for a reduced (first 20 images) and the complete set of training images
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
28
The recognition rate of the geotag propagation system and
Results
The recognition rate of the geotag propagation system and the percentage of the propagated tags versus the threshold involved in user trust model
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
29
The recognition rate of the geotag propagation system
Results
The recognition rate of the geotag propagation system versus the number of tags
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
30
An efficient system for automatic geotag propagation by
Conclusion
An efficient system for automatic geotag propagation by associating locations with distinctive landmarks and using object duplicate detection for tag propagation
User trust information derived from users’ tagging behavior User trust information derived from users tagging behavior for the tag propagation
Only reliable geotags are propagated Increased accuracy of the tag
propagation and a decrease oftagging efforts
The proposed user trust model can begeneralized to photo sharing platformssuch as Panoramio
Multimedia Signal Processing GroupSwiss Federal Institute of Technology
31
Thank you for your attention!
Ivan Ivanov, PhD student, EPFLivan.ivanov@epfl.chivan.ivanov@epfl.ch
Multimedia Signal Processing GroupSwiss Federal Institute of Technology