Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group...
Transcript of Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group...
![Page 1: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/1.jpg)
Deep Label Distribution Learning for Apparent Age Estimation
presented by Bin-Bin Gao
ChaLearn Looking at People: Workshop and Competitions@ICCV, 2015
Xu Yang, Bin-Bin Gao, Chao Xing, Zeng-Wei Huo, Xiu-Shen Wei, Ying Zhou,
Jianxin Wu, and Xin Geng
Dec. 12, 2015 Santiago de Chile
![Page 2: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/2.jpg)
Why is it difficult?
How old do these people look like?
A1: 30 or 32 years old;A2: Around 31 years old;......
A1: 18 or 20 years old;A2: May be 20 years old;......
• It is difficult to provide an exact answer.
![Page 3: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/3.jpg)
Why is it difficult?
ChaLearn Competition- Ages: from 1 to 85 - Training: 2,476 images- Validation: 1,036 images
Public age datasetMorph Album 2- Ages: from 15 to 77 - 55,134 images total
• It is difficult to collect a sufficient and complete training dataset.
• Training data always has small scale and imbalance.
![Page 4: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/4.jpg)
Some potential applications
Cigarette vending machine
http://www.dailymail.co.uk/sciencetech/article-2079048/Kraft-unveils-adults-vending-machine-scans-faces-ensure-children-free-pudding.html
Kraft’s vending machine
• Vending machines can prevent minors buying cigarettes and alcohol by estimating costumer’s apparent age.
• Although the task is very challenging, it has many potential applications.
![Page 5: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/5.jpg)
Many existing methods
Hand-crafted feature
Feature
extractor
Training
classifier
• BIF feature [Guo et al., CVPR 2009]
• OHRank [Chang et al., CVPR 2011]
• CCA, rCCA and kCCA [Guo et al. FGR 2013]
• IIS-LLD and CPNN [Gen et al., TPAMI 2013]
• ……
![Page 6: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/6.jpg)
Many existing methods
Deep learning
Training
classifier
Low-Level
features
Mid-Level
features
High-Level
features
• Multi-scale CNN [Yi et al., ACCV 2014]
• CNN based regression [Huerta et al., PRL 2015]
• CNN for age group classification [Levi & Hassner, CVPR 2015]
• DLA based on CNN features from different layers [Wang et al., WACV 2015]
• ……
![Page 7: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/7.jpg)
Motivations
30±4.17 31±4.24 32±4.23 33±2.01
Faces with similar ages look alike in terms of facial details such aswrinkles or skin smoothness. In other words, there is a correlationamong neighboring ages at both image and feature level.
How to utilize the correlation?
![Page 8: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/8.jpg)
Motivations
How to utilize the correlation?
• Label Distribution (LD) Learning. But it does not learn the visual representations.
• Generating LD , where
X. Geng, C. Yin, and Z.-H. Zhou. Facial age estimation by learning from label distributions. TPAMI, 35(10):2401–2412, 2013.
![Page 9: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/9.jpg)
Proposed methods
Formally:
• The goal of DLDL is to directly learn a conditional probability
mass function 𝒚 = 𝑝(𝒚|𝑋; 𝜽) from the training set, where 𝜽 is the
parameter of the framework.
Deep Label Distribution Learning (DLDL)
![Page 10: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/10.jpg)
Proposed methods
Learning
Forward:
Backward propagation:
• Objective function with K-L divergence:
• The gradient of the K-L loss function is given by
![Page 11: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/11.jpg)
Our datasets
Internet face images collecting
- A set of age related text enquires:
eg., “20 years old”, “20th birthday” and “age-20” for the age of 20 years.
- We use Google, Bing and Baidu image search.
27,197 images 37,606 images
![Page 12: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/12.jpg)
The face image pre-processing
(a) Input (b) Detection (c) Facial points (d) Alignment
M. Mathias et al., Face detection without bells and whistles. In ECCV, pages 720–735, 2014.Y. Sun et al., Deep convolutional network cascade for facial point detection. In CVPR, pages 3476–3483, 2013
Three steps of the images pre-processing
- Face detection
- Facial points detection
- Face alignment
![Page 13: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/13.jpg)
Model architecture
VGG-16
Fine
-tun
e
Fine-tune
Fine-tune
Fine-tune
Fine-tune
Fine-tune
Fine-tune
Age-Net
Morph
Ense
mb
le
Google data
Train Fine-tune
Train
Train
Fine-tune
Fine-tune
Ense
mb
le
Ense
mb
le
ChaLearn
ChaLearn
ChaLearn
ChaLearn
ChaLearn
Stream1
Stream2
Different data Different loss
Different types of input
• Small the number of filters;• Shallow architecture;• Parameter ReLu;• K-L loss.
![Page 14: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/14.jpg)
Training and prediction details
Training
- Gaussian random initialization at different layers.
: The last three layers The last layer The last layer.
: All layers The last layer.
Prediction
- Different fusion strategy
Early fusion:
:Prediction via measuring distance.
: Averaging estimation distribution.
Late fusion :
Averaging the prediction age of two streams.
![Page 15: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/15.jpg)
Comparison
Mean Error on Validation Set
better
The fusion of the two stream is better than single stream.
0.3534
0.3610
0.3377
0.33
0.33
0.34
0.34
0.35
0.35
0.36
0.36
0.37
Stream1 Stream2 Fusion
![Page 16: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/16.jpg)
Final results
Mean Error on Test Set
better
The 4nd place with 0.3057 performance.
0.305763
0.264975 0.270685
0.294835
0.373352 0.37439
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
SEU-NJU CVL-ETHZ ICT-VIPL WVU-CVL UMD Enjudto
![Page 17: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/17.jpg)
Conclusions
DLDL is an end-to-end learning framework which utilizes the
correlation among neighboring labels in both feature learning and
classifier learning;
DLDL can work when the training set is small.
Ensemble strategy: different dataset, different architecture,
different initialization and different fusion.
![Page 18: Deep Label Distribution Learning for Apparent Age Estimation · • CNN for age group classification [Levi & Hassner, CVPR 2015] • DLA based on CNN features from different layers](https://reader034.fdocuments.in/reader034/viewer/2022050200/5f53a8091367037a8d605038/html5/thumbnails/18.jpg)
Any questions