Deep Representation for Imbalanced Classification...
1
Large-scale CelebA face attribute dataset • 200K celebrity images, each with 40 attributes • Highly imbalanced: average positive class rate 23% • Total accuracy = + + Balanced accuracy = 1 2 + Edge detection on BSDS500 dataset • Retrieve from 2M edge label patches with long-tail distribution Learning Deep Representation for Imbalanced Classification Chen Huang 1,2 , Yining Li 1 , Chen Change Loy 1 , Xiaoou Tang 1 1 The Chinese University of Hong Kong 2 SenseTime Group Limited {chuang, ly015, ccloy, xtang}@ie.cuhk.edu.hk Data imbalance is common in visual classification 1. Motivation Wearing hat Not wearing hat … Minority class Majority class Face attribute example Deep embedding: Class-level cluster- & class-level constraint Study traditional re-sampling and cost-sensitive learning scheme 2. Main Idea Triplet embedding Class 1 minority Class 2 majority Class 2 majority Quintuplet embedding Class 1 minority … Cluster j Cluster 2 Cluster 1 Cluster 1 , + < ( , ( − )) < ( , ( −− )) < ( , ( )) – an anchor + – the anchor’s most distant within-cluster neighbor − – the nearest within-class neighbor of the anchor, but from a different cluster −− – the most distant within-class neighbor of the anchor – the nearest between-class neighbor of the anchor Triple-header hinge loss Network architecture • Equal class re-sampling & class costs assignment in batches Training step 3. Large Margin Local Embedding (LMLE) s.t.: ● Clustering by k-means ● Generate quintuplets from cluster & class membership ● Re-sample batches equally from each class ● Forward their quintuplets to CNN to compute loss ● Back-propagation Feature-based clustering Feature learning/updating Every 5000 iterations CNN CNN CNN CNN CNN Triple-header hinge loss Mini- batches Training samples … Embedding Quintuplet Shared parameters Large margin cluster-wise kNN: fast & imbalance resistant 4. Cluster-wise kNN search 10 -8 -6 -4 -2 0 2 4 6 8 -8 -6 -4 -2 0 2 4 6 8 - 1 1 Class 1: cluster 1 Class 1: cluster 2 Class 2: cluster 1 Class 2: cluster 2 Class 2: cluster 3 Class 2: cluster 4 Class 2: cluster 5 15 -10 -5 0 5 10 C2 C3 C4 C5 C1 C2 C1 -8 DeepID2 Triplet Our LMLE 5. Results Total acc. Balanced acc. Triplet-kNN 83 72 Anet 87 80 LMLE-kNN 90 84 Ground truth SketchToken ODS 0.73 DeepContour ODS 0.76 LMLE-kNN ODS 0.78 6. Conclusion Cluster- & class-level quintuplets preserve both locality across clusters and discrimination between classes, irrespective of class imbalance Large margin classification by fast cluster-wise kNN search
Transcript of Deep Representation for Imbalanced Classification...
Large-scale CelebA face attribute dataset
• 200K celebrity images, each with 40 attributes
• Highly imbalanced: average positive class rate 23%
• Total accuracy =𝑡𝑝 + 𝑡𝑛
𝑁𝑝 + 𝑁𝑛Balanced accuracy =
1
2
𝑡𝑝
𝑁𝑝+
𝑡𝑛
𝑁𝑛
Edge detection on BSDS500 dataset
• Retrieve from 2M edge label patches with long-tail distribution
Learning Deep Representation for Imbalanced ClassificationChen Huang1,2, Yining Li1, Chen Change Loy1, Xiaoou Tang1
1The Chinese University of Hong Kong 2SenseTime Group Limited
{chuang, ly015, ccloy, xtang}@ie.cuhk.edu.hk
Data imbalance is common in visual classification
1. Motivation
Wearing
hat
Not wearing
hat
…
Minority class
Majority class
Face attribute example
Deep embedding: Class-level cluster- & class-level constraint
Study traditional re-sampling and cost-sensitive learning scheme
2. Main Idea
Triplet embedding
Class 1
minority
Class 2
majority
Class 2 majority
Quintuplet embedding
Class 1 minority
…
Cluster j
Cluster 2
Cluster 1 Cluster 1
𝐷 𝑓 𝑥𝑖 , 𝑓 𝑥𝑖𝑝+
< 𝐷(𝑓 𝑥𝑖 , 𝑓(𝑥𝑖𝑝−))
< 𝐷(𝑓 𝑥𝑖 , 𝑓(𝑥𝑖𝑝−−
)) < 𝐷(𝑓 𝑥𝑖 , 𝑓(𝑥𝑖𝑛))
𝑥𝑖 – an anchor
𝑥𝑖𝑝+
– the anchor’s most distant within-cluster neighbor
𝑥𝑖𝑝−
– the nearest within-class neighbor of the anchor, but from a different cluster
𝑥𝑖𝑝−−
– the most distant within-class neighbor of the anchor
𝑥𝑖𝑛 – the nearest between-class neighbor of the anchor
Triple-header hinge loss
Network architecture
• Equal class re-sampling & class costs assignment in batches
Training step
3. Large Margin Local Embedding (LMLE)
s.t.:
● Clustering by k-means
● Generate quintuplets from
cluster & class membership
● Re-sample batches equally from each class
● Forward their quintuplets to CNN to
compute loss
● Back-propagation
Feature-based clustering
Feature learning/updating
Every 5000 iterations
CNN
CNN
CNN
CNN
CNN
Trip
le-h
ead
er h
ing
e lo
ss
Mini-
batches
Training
samples …
EmbeddingQuintuplet
Shared parameters
Large margin cluster-wise kNN: fast & imbalance resistant
4. Cluster-wise kNN search
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
Class 1: cluster 1
Class 1: cluster 2
Class 2: cluster 1
Class 2: cluster 2
Class 2: cluster 3
Class 2: cluster 4
Class 2: cluster 5
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
-8 -6 -4 -2 0 2 4 6 8-5
-4
-3
-2
-1
0
1
2
3
4
5
-15 -10 -5 0 5 10-20
-15
-10
-5
0
5
10
15
-10 -8 -6 -4 -2 0 2 4 6 8-8
-6
-4
-2
0
2
4
6
8
NC2
NC3
NC4
NC5
PC1
PC2
NC1
DeepID2 Triplet Our LMLE
5. Results
Total acc. Balanced acc.
Triplet-kNN 83 72
Anet 87 80
LMLE-kNN 90 84
Ground truthSketchToken
ODS 0.73
DeepContour
ODS 0.76
LMLE-kNN
ODS 0.78
6. Conclusion
Cluster- & class-level quintuplets preserve both locality
across clusters and discrimination between classes,
irrespective of class imbalance
Large margin classification by fast cluster-wise kNN search
