Fame cvpr
-
Upload
bilkent-university -
Category
Science
-
view
442 -
download
2
Transcript of Fame cvpr
![Page 1: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/1.jpg)
+FAME:
Face Association through Model Evolution
Pinar Duygulu (CMU, Hacettepe University)Eren Golge (Bilkent University)
![Page 2: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/2.jpg)
+Yale dataset:10 subjects, 9 poses, 64 illuminations
Labelled Faces in the wild13323 Faces of 5749 celebritiesPubFig dataset60,000 images of 200 people
Social Face Classification 4.4 million labeled faces from4,030 people
![Page 3: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/3.jpg)
+Labeling for how many?
![Page 4: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/4.jpg)
+Search web for faces of a query name
![Page 5: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/5.jpg)
+Use this set to learn models
![Page 6: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/6.jpg)
+Variations and sub-categories
![Page 7: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/7.jpg)
+Irrelevant people
![Page 8: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/8.jpg)
+Find category related images in the set of weakly labeled images
![Page 9: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/9.jpg)
+Our previous work: Densest component Most similar set of faces as a subgraph
Assumption:
The most similar subset of faces among the faces associated with a name will be the correct faces
Drawback:
Finds a single subset
Ozkan, D., Duygulu, P., ”Interesting Faces: A Graph Based Approach for Finding People in News”, Pattern Recognition, 2010Ozkan, D., Duygulu, P., ”A Graph Based Approach for Naming Faces in News Photos”, CVPR, 2006Ozkan, D., Duygulu, P., ”Finding People Frequently Appearing in News”, CIVR, 2006
![Page 10: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/10.jpg)
+Our previous work: Concept Maps Grouping and outlier removal
Golge, E., Duygulu, P., “Concept Maps: Mining Noisy Web Data for Concept Learning ”, accepted to ECCV 2014
![Page 11: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/11.jpg)
+Our previous work: Concept Maps Grouping and outlier removal
Assumption:
Faces of a single person can have sub-categories
Outliers are different than the queried person
Drawback:
Eliminates strange looks not in groups as well
![Page 12: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/12.jpg)
+
FAME Face Association through Model Evolution
Capture discriminative and representative category images through iterative data cleansing
Separate category instances versus random images.
Agnostic data refining method against Irrelevancy.
Evade Sub-Grouping using very high dimensional representations.
![Page 13: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/13.jpg)
+Overview of FAME
First discern category candidates (CC) from random set (RS). Define category references(CR) inside CC . Second discern CR from CC. Define spurious instances (SI) against CR and eliminate. Re-Iterate
![Page 14: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/14.jpg)
+Step 1
Discerning category from random set Learn a linear model M1 betweencategory
candidates CC and random set RS. Take the most confidently classified
instances as the category references CR.
![Page 15: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/15.jpg)
+Step 2
Discerning category references from others Another model M2 between category references CR and other category candidates.
![Page 16: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/16.jpg)
+Step 3
Define spurious instances SI against category references CR.
Eliminate SI.
![Page 17: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/17.jpg)
+FAME
![Page 18: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/18.jpg)
+Eliminations by iteration
![Page 19: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/19.jpg)
+Eliminations by iteration
![Page 20: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/20.jpg)
+High Dimensional Representation
High dimensions help a category linearly separable from others despite of category modularity.
![Page 21: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/21.jpg)
+Feature learning
Coates, Adam, Andrew Y. Ng, and Honglak Lee. "An analysis of single-layer networks in unsupervised feature learning." International Conference on Artificial Intelligence and Statistics. 2011.
![Page 22: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/22.jpg)
+
Raw pixel LBP encoded outliers
![Page 23: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/23.jpg)
+Implementation details
FAME Data refining : L1 Logistic Regression with Gauss-Seidel algorithm
[1] Final Classifier: L1 Linear SVM with Grafting[2]. At each iteration 5 images are eliminated.
Feature Learning Augment train data with horizontally flipped images. Re-size each gray-level image 60px height. Contrast Normalization to random patches. ZCA whitening with Ɛ=0.5. Receptive field (patch) size 6x6 pixels 1 pixel stride with k=2400 words. Final feature vector has 5x2400 dimensions.
[1] Shirish Krishnaj Shevade and S Sathiya Keerthi. A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics,19(17):2246–2253, 2003.[2] Simon Perkins, Kevin Lacker, and JFAMEs Theiler. Grafting: Fast, incremental feature selection by gradient descent in function space. The Journal of Machine Learning Research, 3:1333–1356, 2003.
![Page 24: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/24.jpg)
+Datasets
PubFig83 Subset of PubFig with 83 celebrities at least 100 images for each.
N. Pinto, Z. Stone, T. Zickler, and D. Cox, “Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2011.
![Page 25: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/25.jpg)
+Datasets
FAN-Large EASY subset: faces larger than 60x70 px, 138
categories. ALL: no constraint, 365 categories.
M. Ozcan, J. Luo, V. Ferrari, and B. Caputo, “A large-scale database of images and captions for automatic face naming.,” in BMVC, pp. 1–11, 2011.
![Page 26: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/26.jpg)
+ Results on PubFig83
N. Pinto, Z. Stone, T. Zickler, and D. Cox, “Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on, pp. 35–42, IEEE, 2011
B. C. Becker and E. G. Ortiz, “Evaluating open-universe face identification on the web,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 904–911, IEEE, 2013.
No data refining, only our classification pipeline.Models are trained on the training set of the given dataset~5% improvement on State of Art
![Page 27: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/27.jpg)
+Evaluations
![Page 28: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/28.jpg)
+False versus true outlier elimination
![Page 29: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/29.jpg)
+Cross validation accuracies
![Page 30: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/30.jpg)
+Number of eliminations versus accuracy
![Page 31: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/31.jpg)
+Models learned from weakly labeled set
Baseline: all images collected for the query are used AME-M1 : Only M1 classifier which removes against
global negatives AME-SVM : with SVM as the final classifier AME-LR : the proposed method
S. Singh, A. Gupta, and A. A. Efros, “Unsupervised discovery of mid-level discriminative patches,” in European Conference Computer Vision (ECCV), 2012.
![Page 32: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/32.jpg)
+Summary
A method to build training sets from weakly-labeled images
Iterative pruning removes the outliers which are the least confident instances
High dimensional feature representation handles the variations
![Page 33: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/33.jpg)
+
TUBITAK 112E174
CHIST-ERA MUCKE
US Department of Defense, U. S. Army Research Office (W911NF-13-1-0277)
National Science Foundation Grant No. IIS-1251187
Thanks
![Page 34: Fame cvpr](https://reader034.fdocuments.in/reader034/viewer/2022042701/55bec748bb61eb1e5c8b47eb/html5/thumbnails/34.jpg)
+Use annotated control set as a start point.
Fergus et. al. [1], OPTIMOL, Li and Fei-Fei [2] We use fully autonomous framework.
Use Textual Captions Berg and Forsyth [3] We use only visual content
Discriminative image cues Efros et al. [4] “Discriminative Patches”, Q. Li et al.[5]
We use single computer with faster and better results.
[1] Fergus, R., Fei-Fei, L., Perona, P., Zisserman, A.: Learning object categories from google’s image search. In: Computer Vision, 2005. ICCV 2005[2] Berg, T.L., Berg, A.C., Edwards, J., Maire, M., White, R., Teh, Y.W., Learned-Miller, E.G., Forsyth, D.A.: NFAMEs and faces in the news. In: IEEE Conference on Computer VisionPattern Recognition (CVPR). Volume 2. (2004) 848–854[3] Li, L.J., Fei-Fei, L.: Optimol: automatic online picture collection via incremental model learning. International journal of computer vision 88(2) (2010) 147–168[4] Li, Q., Wu, J., & Tu, Z. Harvesting Mid-level Visual Concepts from Large-scale Internet Images.