Consistency-based Semi-supervised learning for Object ...Type of object detector: quick review...

CAP 6412 Advanced Computer Vision, Spring 2020

Consistency-based Semi-supervised learning for Object Detection

Authors: Jisoo Jeong, Seungeui Lee, Jeesoo Kim, Nojun Kwak

Venue: Advances in Neural Information Processing Systems 32 (NIPS 2019)

Presenter: Ishan Dave

1

Consistency-based Semi-supervised Learning for Object Detection (NIPS-2019)CAP 6412 Advanced Computer Vision, Spring 2020

Outline of this presentation

● Understanding the problem and terminologies● Existing Approaches to solve the problem● Inspiration of the proposed work● Proposed method● Technical Contribution● Results● Conclusion

2


Understanding the problem and terminologies: Object Detector

Image Courtesy: https://towardsdatascience.com/evolution-of-object-detection-and-localization-algorithms-e241021d8bad

3

Understanding the Problem

Existing methods

Inspiration of the work

Proposed Method

Technical Contribution Results

https://towardsdatascience.com/evolution-of-object-detection-and-localization-algorithms-e241021d8bad


Type of object detector: quick reviewSingle stage Obj Detector

Two stage Obj Detector

Region Proposal Network Classifier

Eg. SSD, YOLO

Eg. Faster RCNN

4

Liu, Wei, et al. "Ssd: Single shot multibox detector." European conference on computer vision. Springer, Cham, 2016.Redmon, Joseph, and Ali Farhadi. "Yolov3: An incremental improvement." arXiv preprint arXiv:1804.02767 (2018).Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE international conference on computer vision. 2015.


Existing methods


Proposed Method



Understanding the problem and terminologies: Motivation

Supervised LearningObject

DetectorModel

Classification and Localization Loss

Semi-Supervised Learning

Object DetectorModel

Classification and Localization Loss for

labeled data

But what losses for Unlabled data?

5


Existing methods


Proposed Method



Any questions till this point?

✓ Understanding the problem and terminologies✓ Object Detector✓ Different Level of Supervision✓ Motivation

❏ Existing Approaches to solve the problem

6


Existing methods


Proposed Method



A common approach in Semi-supervised learning

Object DetectorModel

Classification and Localization Loss

TRAINEDObject

DetectorModel

Labeled data

Prediction“Horse” at [245, 240, 50, 50] with 0.93 confidence

Confidence threshold

Inference Mode

Training Mode

“Pseudo Labels”

“Self-Training approach”

7


Existing methods


Proposed Method



Consistency regularization

Classifier Model

Original (x)

Perturbed (x’)

Loss

f(x)

f(x’)

State-of-the-Art in Semi-Supervised Classification!

8


Existing methods


Proposed Method



Consistency regularization

Localization Model

Original (x)

Perturbed (x’)

f(x)

f(x’)

Hard to find one to one correspondence!

9


Existing methods


Proposed Method


Image courtesy https://www.petmoo.com/dogs/golden-retriever-puppies/

https://www.petmoo.com/dogs/golden-retriever-puppies/


Proposed Work “CSD”

1010


Existing methods


Proposed Method



CSD-Single-Stage Detector

11


Existing methods


Proposed Method



CSD-Two-Stage Detector

12


Existing methods


Proposed Method



Loss function

1313

Overall Loss

Supervised Loss

Classification Localization

Consistency Loss

Classification Consistency

Localization Consistency


Existing methods


Proposed Method



Consistency Loss

For a pair of Bounding Box

Average classification loss for all Bounding box pair

Prediction of Localization:

≆

Average classification loss for all Bounding box pair

Total Consistency Loss

Classification Localization

14


Existing methods


Proposed Method



Overall loss for Object Detector

Weight Scheduling for “Ramp-up and Ramp-down Strategies”

15


Existing methods


Proposed Method



Background EliminationConsistency loss computed with all candidates will be easily dominated by background objects

Image courtesy: https://www.learnopencv.com/wp-content/uploads/2017/10/object-recognition-false-positives-true-positives.jpg

16


Existing methods


Proposed Method


Example:For background object (in blue) the detector predicts some prediction value: “Background” as “Dog”: 0.3, 0.2, 0.3, 0.1, 0.2----> should be ‘0’

Loss: 0.3+0.2+0.3+0.1+0.2= 1.1

“Dog” as “Dog”: 0.9, 0.8---->Should be ‘1’Loss:0.1+0.2= 0.3

https://www.learnopencv.com/wp-content/uploads/2017/10/object-recognition-false-positives-true-positives.jpg

https://www.learnopencv.com/wp-content/uploads/2017/10/object-recognition-false-positives-true-positives.jpg


ExperimentsDatasets:

1. PASCAL VOC: 20 Classesa. VOC07- 5k imagesb. VOC12- 11.5k images

2. MSCOCO: 80 Classes

17


Existing methods


Proposed Method



Experiments

18


Existing methods


Proposed Method


Metric:

Mean Average Precision (mAP): AUC of Precision-Recall curve and take average across classes


Results

1. Using CSD method improves the mAP

2. Combined classification and localization consistency provides better mAP than the individual

3. Combined BE, classification and localization consistency provides the best mAP

4. Improvement in 2-stage object detector is lesser than that of 1-stage object detector

19


Existing methods


Proposed Method



Results-Effect of Unlabeled data with different distribution

2020

20 Classes


Existing methods


Proposed Method



Results: Consistency loss without unlabeled data

2121


Existing methods


Proposed Method



Limitations 1. Not significant improvement on 2-stage Object Detector (~0.8%)

2222

2. Unlabeled data should have a similar distribution to the labeled data


ConclusionUsing Semi-supervised technique- Consistency regularization this paper improves the performance of the object detector using unlabeled data

2323

Consistency-based Semi-supervised learning for Object ...Type of object detector: quick review...

Documents

Transcript of Consistency-based Semi-supervised learning for Object ...Type of object detector: quick review...