Complementing Representation Deficiency in Few-shot Image ...
Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental...
Transcript of Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental...
![Page 1: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/1.jpg)
Kuilin Chen, Chi-Guhn Lee
University of Toronto
Incremental Few-shot Learning via Vector
Quantization in Deep Embedded Space
![Page 2: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/2.jpg)
Incremental few-shot learning
• Incremental learning is a learning paradigm that allows the model
to continually learn new tasks on novel data, without forgetting
how to perform previously learned tasks
• The capability of incrementally learning new tasks without
forgetting old ones is challenging due to catastrophic forgetting
• This challenge becomes greater when novel tasks contain very
few labelled training samples
• It is desirable to develop algorithms to support incremental
learning from very few samples
![Page 3: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/3.jpg)
Unified Framework
• Unified framework of IDLVQ for both classification and regression
can be derived from a Gaussian mixture
• A raw input is projected into a feature space by a neural network
• Reference vectors are place in feature
space
• We will add more reference vectors as we learn novel tasks
• The marginal distribution of a feature vector is a Gaussian mixture
• Assumption: isotropic Gaussian centered at a reference vector
with the same covariance
![Page 4: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/4.jpg)
Unified Framework
• Posterior:
• is a Gaussian kernel
• The conditional expectation of output:
• q is the reference target
• The model is learned by minimizing an appropriate loss function
![Page 5: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/5.jpg)
Incremental few-shot classificationCompact intra-class variation
Update model when necessary
Less forgetting
![Page 6: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/6.jpg)
Visualization of feature space
Standard NN IDLVQCIDLVQ-C w.o. intra loss
![Page 7: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/7.jpg)
Prediction accuracy on CUB all classes using the
10-way 5-shot incremental setting
![Page 8: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/8.jpg)
Ablation study
![Page 9: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/9.jpg)
IDLVQ-R
• For regression tasks, the model is learned by minimizing MSE
• The learnable parameters in the model are:
• The loss is differentiable w.r.t. all parameters and learning is end-
to-end
• It can be interpreted as a sparse kernel smoother
![Page 10: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/10.jpg)
Regression Example
• We generate some nonlinear data
• The old data contains 1000 samples generated when
• The model was originally trained on old data
• 1st novel task: 5-shot samples by sampling
• 2nd novel task: 5-shot samples by sampling
• Test samples are randomly generated by sampling
![Page 11: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/11.jpg)
Result
• (a) Our method
• (b) Fine-tune using
novel data only
• (c) Fine-tune using
novel data and saved
exemplars
• (d) Offline training using
all training samples
from all tasks
![Page 12: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/12.jpg)
Conclusions
• We propose a unified framework to handle incremental few-
shot classification and regression problems
• The proposed method is based on vector quantization in deep
embedded space
• Empirical studies show that the proposed achieves state-of-
the-art performance
![Page 13: Incremental Few-shot Learning via Vector Quantization in ...06-08-00...2000/06/08 · Incremental Few-shot Learning via Vector Quantization in Deep Embedded Space Incremental few-shot](https://reader035.fdocuments.in/reader035/viewer/2022071606/6142acc7b7accd31ec0eda1b/html5/thumbnails/13.jpg)
Thank you!