Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge...
-
Upload
meredith-sutton -
Category
Documents
-
view
218 -
download
1
Transcript of Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge...
![Page 1: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/1.jpg)
1
Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge
Date: 2011/11/21
Source: Claudiu S. Firan (CIKM’10)
Speaker: Er-gang Liu
Advisor: Dr. Jia-ling Koh
![Page 2: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/2.jpg)
2
Motivation
Image processing
Tag
Slow
fast
![Page 3: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/3.jpg)
3
Outline
• Introduction• Data Set• Event Detection• Evaluation Methodology• Experiment• Conclusion
![Page 4: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/4.jpg)
4
Introduction
• Motivation:• A huge amount of digital pictures remains untouched unless
powerful techniques for image indexing and retrieval become available.
• Goal:• To solve some of the users' knowledge management problems by
providing the means to organize and browse content based on events.
![Page 5: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/5.jpg)
5
Introduction• On the domain of pictures, in particular on a subset of
Flickr data.
• Using events as the primary means to organize pictures.
• Construct classifiers for classifying pictures into different event categories.
![Page 6: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/6.jpg)
6
Event Definition• Defining an event as "specific thing happening”
• specific time • specific place
• EX : birthdays, marriages , vacations
• Dimension• Local
• EX : John’s birthday, 11/21 Lab KDD Group Meeting
• Global• EX : 2011 NBA lockout
![Page 7: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/7.jpg)
7
![Page 8: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/8.jpg)
8
Flickr Images
WordNet concepts
Event ids
Data Set
Upcoming DataSet
Wikipedia categories
Event Groups
![Page 9: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/9.jpg)
9
Event Group Wikipedia category WordNet concept
Data Set
![Page 10: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/10.jpg)
10
![Page 11: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/11.jpg)
11
Data Set• Unclustered data set
• Event Groups
• hierarchical clustering• based on Wikipedia categories• based on WordNet concepts
• Forming a three-level hierarchical taxonomy.
• An extensive list of [events], along with the corresponding [categories] and the super-[concepts]
![Page 12: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/12.jpg)
12
• Try to assign the pictures to the corresponding event categories.
• Photo event detection algorithm is a probabilistic classifier trained on the Flickr ground truth using tags as input features.
• Using the Naive Bayes Multinomial implementation • textual { tags, title, description }• Based on time information -- SVM
• time 、 time + textual { tags, title, description }
Event Detection From Tags
![Page 13: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/13.jpg)
13
Event Detection Algorithm
Discard tags freq < 10Discard photos tag Discard class photo <100
![Page 14: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/14.jpg)
14
= {1, 1, 0, 0}
= {0, 0, 1, 1}
= {1, 1, 1, 0}
= {1, 0, 0, 1}
Train Data Class= Sports
Test Data
Yes
No
Yes
Yes
t1 t2 t3 t4
ti=1 | YES 2/3 2/3 2/3 1/3
ti=0 | YES 1/3 1/3 1/3 2/3
ti=1 | No 1/2 1/2 0 1
ti=0 | No 1/2 1/2 1 0
= {0, 1, 0, 1} No
= {1, 0, 1, 0} ?
P(E | sports = yes) P( sports = yes ) = ( 0.0593
P(E | sports = no) ( sports = no ) = ( )
Event Detection Algorithm
![Page 15: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/15.jpg)
15
• Evaluation methodology• Accuracy• Precision• Recall• NMI• B-Cubed
Evaluation Methodology
![Page 16: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/16.jpg)
16
Evaluation
PREDICTED CLASS
ACTUALCLASS
+ -
+ 150 40
- 60 250 %9.7840150
150 Recall
%4.7160150
150 Precision
%80500
250150 Accuracy
![Page 17: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/17.jpg)
17
Evaluation - B-Cubed
%75503.1
1.127
0.789+0.714
0.789*0.714*2 Cubed-B
![Page 18: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/18.jpg)
18
Evaluation – NMI
NMI = 1 NMI = 0
![Page 19: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/19.jpg)
19
e1 e2
c1 150 40
c2 60 250
I(C,E) = ( log ) + ( log ) + ( log ) + ( log )
≈ ( 0.082 ) + ( -0.04 ) + ( -0.035 )+ ( 0.072 )
≈ 0.079
Evaluation – NMI
![Page 20: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/20.jpg)
20
Evaluation – NMI
H(C) = - ( log ) - (log ) ≈ - ( -0.160) - ( -0.129) ≈ 0.289
H(E) = - ( log ) - ( log) ≈ - ( -0.158 ) - ( -0.137) ≈ 0.297
e1 e2
c1 150 40
c2 60 250
![Page 21: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/21.jpg)
21
Evaluation – NMI
0.270.293
0.079
2 / 0.297)+(0.289
0.079 NMI
![Page 22: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/22.jpg)
22
Experiment
![Page 23: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/23.jpg)
23
Local vs. Global Events
![Page 24: Bring Order to Your Photos: Event-Driven Classification of Flickr Images Based on Social Knowledge Date: 2011/11/21 Source: Claudiu S. Firan (CIKM’10)](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ceb5503460f949b6289/html5/thumbnails/24.jpg)
24
Conclusion• The approach is not restricted to this collection, but being
applicable to any other photo set or other types of multimedia content (e.g. videos, music, etc.) containing similar metadata.
• Show that photo event-based classification is feasible and confirm once more the quality of the user provided tags.