SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning
1
SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning Introduction People who use album on sharing websites always like their photos to be: Popular : more people view their photos & good appreciation Easy management : while attach to groups & add tags are big trouble (about 15000 groups related to “dog” on Flickr) Our Goal: recommend photo(s) to suitable and popular groups and attach relevant tags to each photo automatically Prediction result ● Photo-level data collection Group-level data collection (b) Training Data Acquisition Feature extraction (d) Concept Detection SVM training (c) Model Learning Flickr photo Database concept detector Feature extraction top-n concepts c 1 c n … g 1,1 g 1,p … g n,p g n,1 … (e) Group Recommendation (f) Tag Recommendation Input test image … T 1,1 T 1,2 T 1,d … T n,1 T n,2 T n,d … Recommend to user Recommend to user (a) Concept Definition -- “dog”, “tiger”, “flower”, … Concept Detection Compare two source of pseudo-positive training data from Flickr Photo-Level data mechanism Group-Level data mechanism Provide a new idea of “how to acquire reliable search-based data ” The SVM predictor gives each concept a probability value to indicate the degree that the input photo fits this concept. For the top-n concepts, we recommend users the most popular groups and tags related to these concepts using our ranking algorithm. Animal Architect ure Nature Scene Portrai t Plant Color Oriente d Overall Average Photo level 1.10 1.51 1.72 1.07 1.79 1.51 1.55 Group level 1.24 1.68 1.90 1.40 1.92 1.56 1.69 tiger cat animals dog monkey snake rabbit portrait bird horse 0% 4% 8% 12% 16% 20% Photo-level probability distribution average tiger animals dog cat snake monkey portrait rabbit bird wood 0% 4% 8% 12% 16% 20% Group-level probability distribution average Subjective test result The score for the top-3 concepts recommendation results [1]: S = (P*1 + R*0.5 +W*0)/N c (N c = 15) [1] L. S. Kennedy, S.-F. Chang, and I. V. Kozintsev, "To search or to label?: predicting the performance of search-based automatic image classifiers," in Proc. ACM MIR’06, pp.249 – 258, 2006. Hong-Ming Chen, Ming-Hsiu Chang, Ping-Chieh Chang, Ming-Chun Tien, Winston H. Hsu, and Ja-Ling Wu Communications and Multimedia Lab, National Taiwan University {blacksmith,cmhsiu,pingchieh,trimy,winston,wjl}@cmlab.csie.n tu.edu.tw
description
SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning. Input test image . (c) Model Learning . (d) Concept Detection . SVM training . Feature extraction . concept detector . - PowerPoint PPT Presentation
Transcript of SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning
SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning
IntroductionPeople who use album on sharing websites always like their photos to be:
Popular : more people view their photos & good appreciation Easy management : while attach to groups & add tags are big trouble(about 15000 groups related to “dog” on Flickr)Our Goal:recommend photo(s) to suitable and popular groups and attach relevant tags to each photo automatically
Prediction result ●
Photo-level data collection
Group-level data collection
(b) Training Data Acquisition
Feature extraction
(d) Concept Detection
SVM training
(c) Model Learning
Flickr photo Database
concept detector
Feature extraction
top-n concepts
c1 cn…
g1,1 g1,p… gn,pgn,1 …
(e) Group Recommendation
(f) Tag Recommendation
Input test image
…T1,1
T1,2
T1,d
… Tn,1
Tn,2
T n,d
…
Recommend to user
Recommend to user
(a) Concept Definition -- “dog”, “tiger”, “flower”, …
Concept DetectionCompare two source of pseudo-positive training data from Flickr Photo-Level data mechanism Group-Level data mechanismProvide a new idea of “how to acquire reliable search-based data ”
The SVM predictor gives each concept a probability value to indicate the degree that the input photo fits this concept. For the top-n concepts, we recommend users the most popular groups and tags related to these concepts using our ranking algorithm.
Animal ArchitectureNature Scene Portrait Plant
Color Oriented
Overall Average
Photo level 1.10 1.51 1.72 1.07 1.79 1.51 1.55
Group level 1.24 1.68 1.90 1.40 1.92 1.56 1.69
tiger cat animals dog monkey snake rabbit portrait bird horse0%
4%
8%
12%
16%
20%Photo-level probability distribution average
tiger animals dog cat snake monkey portrait rabbit bird wood0%
4%
8%
12%
16%
20%
Group-level probability distribution average
Subjective test result The score for the top-3 concepts recommendation results [1]: S = (P*1 + R*0.5 +W*0)/Nc (Nc = 15)
[1] L. S. Kennedy, S.-F. Chang, and I. V. Kozintsev, "To search or to label?: predicting the performance of search-based automatic image classifiers," in Proc. ACM MIR’06, pp.249 – 258, 2006.
Hong-Ming Chen, Ming-Hsiu Chang, Ping-Chieh Chang, Ming-Chun Tien, Winston H. Hsu, and Ja-Ling WuCommunications and Multimedia Lab, National Taiwan University{blacksmith,cmhsiu,pingchieh,trimy,winston,wjl}@cmlab.csie.ntu.edu.tw
