Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf ·...
Transcript of Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf ·...
![Page 1: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/1.jpg)
Region-Based Active Learning for Efficient Labelling in Semantic Segmentation
Tejaswi Kasarla1, G Nagendar1, Guruprasad Hegde2, Vineeth N. Balasubramanian3, C.V. Jawahar1
1 Center for Visual Information Technology, IIIT Hyderabad, India2 Bosch Research and Technology Centre, India3 IIT Hyderabad, India
![Page 2: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/2.jpg)
Annotations for Segmentation
~ 1.5 to 2 hours for fine annotation
Expensive to obtain!
[1] Cordts el. al, Cityscapes Dataset, 2016
1024
x 2
048
![Page 3: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/3.jpg)
Getting less expensive annotation
The way we annotate matters!
Sampling Strategy 1
Sampling Strategy 2
Our sampling strategy
![Page 4: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/4.jpg)
Getting less expensive annotation
What if we have a way to get similar performance of full annotations with intelligently selecting the data points?
Active Learning is the answer![2] Burr Settles, Active Learning Literature Survey, 2009
Perf
orm
ance
(IO
U)
Percentage pixel annotation
0.93x
0.94x
0.97x
0.95x
Fully Supervised performance (1x)
![Page 5: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/5.jpg)
Method
But how to intelligently select the data points?
Entropy:
By finding the uncertain regions in
the image and providing
annotation for it.
![Page 6: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/6.jpg)
Pipeline
Unlabeled image(s)
Initial Segmentation Result(s)
ICNet
Oracle/Human Annotator
Updated labels for selected pixels
Previously trained model
Final Segmentation Result(s)
Pixels selected for annotation
Picking the informative pixels using active learning techniques
![Page 7: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/7.jpg)
Proposed Active Learning Method
● Pixel - Obtain pixel entropies to pick the most m% uncertain pixels to query for annotation.
● Edge + Pixel - Gives more weightage to pick edge pixels and then picks most uncertain pixels
● SP - region based method – superpixels are annotated (instead of pixels)
● SP + CRF - CRF post-processing after annotating superpixel
● Class-specific SP + CRF - SP+CRF for each class separately
![Page 8: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/8.jpg)
Results
Datasets for evaluation: Cityscapes, Mapillary
DNN for the training: ICNet
Experimental Setting: Cityscapes - 1175 fully annotated images (to train initial model) + 1800 unlabeled images (to be used for partial annotation with active learning)Mapillary - 18000 unlabeled images
![Page 9: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/9.jpg)
Results: Cityscapes
Performance of the proposed active learning methods over incremental selection of batches on cityscapes data.
![Page 10: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/10.jpg)
Sampled active learning (super) pixels and their corresponding segmentation results of various methods after training.
Image Random Entropy Entropy+Edge SP+CRF
10% annotated pixels
Segmentation ResultsGround truth
Results: Cityscapes
![Page 11: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/11.jpg)
Results: Mapillary
Performance of the proposed active learning methods over incremental selection of batches on mapillary data.
![Page 12: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/12.jpg)
Results: Mapillary
Groundtruth Using 10% labelling
No additional labels
Image
Segmentation results using transfer learning on mapillary data.
![Page 13: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/13.jpg)
Qualitative Results
The results of the proposed region-based active learning method and the fully supervised method are visually almost similar.
Proposed Method Fully Supervised Method
![Page 14: Region-Based Active Learning for Efficient Labelling in ... spotlight presentation.pdf · Region-Based Active Learning for Efficient Labelling in Semantic Segmentation Tejaswi Kasarla1,](https://reader033.fdocuments.in/reader033/viewer/2022050402/5f805da90decd4736e6d66bb/html5/thumbnails/14.jpg)
Thank you.