Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse...
Transcript of Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse...
![Page 1: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/1.jpg)
Unsupervised Feature Learning by Deep Sparse Coding
3/1/20 1
Yunlong He, Georgia TechKoray Kavukcuoglu, DeepMind TechnologiesYun Wang, Princeton UniversityArthur Szlam, The City College of New YorkYanjun Qi, University of Virginia
![Page 2: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/2.jpg)
Two lines of Previous Research• Bag of Visual words
• Deep learning
• Motivation: Can we combine the power of two methods?
3/1/20 2
Image Dense CodePyramid Pooling ClassifierSparse Code
Input Image Features
Encoder
DecoderFeatures
Encoder
Decoder
Class label
Encoder
Decoder…
Repeat[Hinton & Salakhutdinov 06, 09]
Layer 1 Layer n [LeCun et al. 98]
[Lazebnik et al. 06][Yang et al. 09]
SIFT Descriptors
![Page 3: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/3.jpg)
Background: Bag of Visual words
3/1/20 3
Image
Dense Code
Pyramid Pooling
Classifier
Sparse Code
![Page 4: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/4.jpg)
Background: Bag of Visual words
3/1/20 4
Image
SIFT Descriptors
Image
Dense Code
![Page 5: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/5.jpg)
Background: Bag of Visual words
3/1/20 5
Example:900 Patches
Image
Dense Code
![Page 6: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/6.jpg)
Background: Bag of Visual words
3/1/20 6
SIFT filter
SIFT Descriptorsfor every patch
Image
Dense Code
![Page 7: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/7.jpg)
Background: Bag of Visual words
3/1/20 7
Dense Code
Sparse Code
![Page 8: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/8.jpg)
Background: Bag of Visual words
3/1/20 8
Pyramid Pooling
Sparse Code
Level 1 Level 0
21 pooling regions
Concatenate Sparse Codes of 21 pooling regions
![Page 9: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/9.jpg)
Background: Bag of Visual Words
Relatively shallow
3/1/20 9
Image
Dense Code
Pyramid Pooling
Classifier
Sparse CodeCan we learn multiple layers of sparse representations?
![Page 10: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/10.jpg)
Deep Sparse Coding
Relatively shallow
3/1/20 10
Image
Dense Code
Pyramid Pooling
Classifier
Sparse Code
Image
Dense Code
Pyramid PoolingSparse Code
Dense Code
Sparse Code
Dense Code
Sparse Code
Pyramid Pooling
Pyramid Pooling
Classifier
Unsupervised Deep Architecture!
![Page 11: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/11.jpg)
Deep Sparse Coding
Relatively shallow
3/1/20 11
Image
Dense Code
Pyramid Pooling
Classifier
Sparse Code
Image
Dense Code
Pyramid PoolingSparse Code
Dense Code
Sparse Code
Dense Code
Sparse Code
Pyramid Pooling
Pyramid Pooling
Classifier
Unsupervised Deep Architecture!
![Page 12: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/12.jpg)
Deep Sparse Coding
Relatively shallow
3/1/20 12
Image
Dense Code
Pyramid Pooling
Classifier
Sparse Code
Image
Dense Code
Pyramid PoolingSparse Code
Dense Code
Sparse Code
Dense Code
Sparse Code
Pyramid Pooling
Pyramid Pooling
Classifier
Unsupervised Deep Architecture!
![Page 13: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/13.jpg)
Dimension expansion
Idea:• Separation (for classification)• Representation (bag of visual words)
Method:• Sparse coding
3/1/20 13
Dense Code
Sparse Code
![Page 14: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/14.jpg)
Dimension reduction
Idea:• Invariance• Compositionality
Method:• Locally spatial pooling + Low dimensional embedding• Both unsupervised
3/1/20 14
Sparse Code
Dense Code
![Page 15: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/15.jpg)
Locally spatial pooling
3/1/20 15
Covers larger area
![Page 16: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/16.jpg)
Low dimensional embedding
3/1/20 16
embedding
Unstable Stable
![Page 17: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/17.jpg)
Low dimensional embedding
3/1/20 17
embedding
Unstable Stable
Idea: embedding with the help of spatial information
![Page 18: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/18.jpg)
Low dimensional embedding
3/1/20 18
embedding
Dimensionality Reduction by Learning an Invariant Mapping (DR. LIM)
![Page 19: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/19.jpg)
Low dimensional embedding
3/1/20 19
embedding
Dimensionality Reduction by Learning an Invariant Mapping (DR. LIM)
![Page 20: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/20.jpg)
Deep Sparse Coding
Relatively shallow
3/1/20 20
Image
Dense Code
Pyramid Pooling
Classifier
Sparse Code
Image
Dense Code
Pyramid PoolingSparse Code
Dense Code
Sparse Code
Dense Code
Sparse Code
Pyramid Pooling
Pyramid Pooling
Classifier
Unsupervised Deep Architecture!
![Page 21: Unsupervised Feature Learning by Deep Sparse Coding · Unsupervised Feature Learning by Deep Sparse Coding 3/1/20 1 YunlongHe, Georgia Tech KorayKavukcuoglu,DeepMindTechnologies Yun](https://reader030.fdocuments.in/reader030/viewer/2022040906/5e7ac5ca8469660aeb734b2b/html5/thumbnails/21.jpg)
3/1/20 21
Conclusion
§Combine sparse coding with deep learning§Combine dimension expansion and dimension reduction§A connecting function is learned by embedding method§ Combining multiple layers of sparse code achieves state-of-the-art performance on image classification tasks