Alex Kendall - Model Uncertainty in Deep …...2. Alex Kendall, Vijay Badrinarayanan and Roberto...

Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla http://mi.eng.cam.ac.uk/projects/segnet/ References 1. Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. PAMI, 2017. 2. Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla. Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. BMVC, 2017. 3. Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint arXiv:1703.04977, 2017. 4. Alex Kendall, Yarin Gal and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. arXiv preprint arXiv:1705.07115, 2017. Insights • We can obtain per-class model uncertainty estimates for scene understanding models • Bayesian inference more important in late encoder and early decoder layers • Improves segmentation performance by 2-3% across popular models • MC dropout outperforms weight averaging after 6 samples and converges after 40 samples • Especially effective for small datasets (e.g. CamVid) • Model uncertainty increases for rare and difficult classes • Model uncertainty is useful for safe autonomous decision making, active learning and label propagation Further Applications • Distinguish Aleatoric (sensor) uncertainty and Epistemic (model) uncertainty [3] • Use uncertainty to improve multi-task learning [4] • Semantic segmentation, instance segmentation and depth regression from a single input image [4] Model Standard Bayesian DilationNet 71.3% 73.1% FCN 62.2% 65.4% SegNet 59.1% 60.5% Convolutional Encoder-Decoder Input Segmentation Model Uncertainty Stochastic Dropout Samples Conv + Batch Normalisation + ReLU Dropout Pooling/Upsampling Softmax mean variance RGB Image We use Monte Carlo dropout sampling at test time to generate a posterior distribution of pixel class labels. Bayesian SegNet architecture PASCAL VOC 2012 Test Server Performance Input Image Ground Truth Model Segmentation Aleatoric Uncertainty Epistemic Uncertainty

Upload
others
Category

Documents
view
2
download
0

Embed Size (px):

Transcript of Alex Kendall - Model Uncertainty in Deep …...2. Alex Kendall, Vijay Badrinarayanan and Roberto...

Page 1: Alex Kendall - Model Uncertainty in Deep …...2. Alex Kendall, Vijay Badrinarayanan and Roberto Cipolla. Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures

BayesianSegNet:ModelUncertaintyinDeepConvolutionalEncoder-DecoderArchitecturesforSceneUnderstandingAlexKendall,VijayBadrinarayananandRobertoCipolla

http://mi.eng.cam.ac.uk/projects/segnet/

References1. VijayBadrinarayanan,AlexKendall,RobertoCipolla.SegNet:ADeepConvolutionalEncoder-DecoderArchitectureforImageSegmentation. PAMI,2017.2. AlexKendall,VijayBadrinarayananandRobertoCipolla.BayesianSegNet:ModelUncertaintyinDeepConvolutionalEncoder-DecoderArchitecturesfor

SceneUnderstanding.BMVC,2017.3. AlexKendallandYarinGal.WhatUncertaintiesDoWeNeedinBayesianDeepLearningforComputerVision?arXivpreprintarXiv:1703.04977,2017.4. AlexKendall,YarinGalandRobertoCipolla.Multi-TaskLearningUsingUncertaintytoWeighLossesforSceneGeometryandSemantics.arXivpreprint

arXiv:1705.07115,2017.

Insights• Wecanobtainper-classmodeluncertaintyestimatesforsceneunderstandingmodels• Bayesianinferencemoreimportantinlateencoderandearlydecoderlayers• Improvessegmentationperformanceby2-3% acrosspopularmodels• MCdropoutoutperformsweightaveragingafter6samplesandconvergesafter40samples• Especiallyeffectiveforsmalldatasets(e.g.CamVid)• Modeluncertaintyincreasesforrareanddifficultclasses• Modeluncertaintyisusefulforsafeautonomousdecisionmaking,activelearningandlabelpropagation

FurtherApplications• DistinguishAleatoric (sensor)

uncertaintyandEpistemic (model)uncertainty[3]

• Useuncertaintytoimprovemulti-tasklearning[4]

• Semanticsegmentation,instancesegmentationanddepth regressionfromasingleinputimage[4]

Model Standard Bayesian

DilationNet 71.3% 73.1%

FCN 62.2% 65.4%

SegNet 59.1% 60.5%

Convolutional Encoder-DecoderInputSegmentation

Model Uncertainty

Stochastic DropoutSamples

Conv + Batch Normalisation + ReLUDropout Pooling/Upsampling Softmax