Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf ·...
Transcript of Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf ·...
![Page 1: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/1.jpg)
AlgorithmicIntelligenceLab
AlgorithmicIntelligenceLab
EE807:RecentAdvancesinDeepLearningLecture13
Slidemadeby
Kimin LeeKAISTEE
NoveltyDetectionforDeepClassifiers
![Page 2: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/2.jpg)
AlgorithmicIntelligenceLab
1. Introduction• Whatisnoveltydetection?• Overview
2. UtilizingthePosteriorDistribution• Baselinemethod• Post-processingmethod
3. UtilizingtheHiddenFeatures• Localintrinsicdimensionality• Mahalanobis distance-basedscore
TableofContents
2
![Page 3: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/3.jpg)
AlgorithmicIntelligenceLab
1. Introduction• Whatisnoveltydetection?• Overview
2. UtilizingthePosteriorDistribution• Baselinemethod• Post-processingmethod
3. UtilizingtheHiddenFeatures• Localintrinsicdimensionality• Mahalanobis distance-basedscore
TableofContents
3
![Page 4: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/4.jpg)
AlgorithmicIntelligenceLab
• Deepneuralnetworks(DNNs)canbegeneralizedwell whenthetestsamplesarefromsimilardistribution(i.e.,in-distribution)
WhatisNoveltyDetection?
4
Trainingdata=animal
TestsampleDNNs
Softmax
cat dog
0.99
![Page 5: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/5.jpg)
AlgorithmicIntelligenceLab
• Deepneuralnetworks(DNNs)canbegeneralizedwell whenthetestsamplesarefromsimilardistribution(i.e.,in-distribution)
• However,intherealworld,therearemanyunknownandunseensamples thatclassifiercan’tgivearightanswer
WhatisNoveltyDetection?
5
Trainingdata=animal
TestsampleDNNs
Softmax
cat dog
0.99
Unseensample,i.e.,out-of-distribution(notanimal)
Unknownsample Adversarialsamples[Goodfellow etal.,2015]
![Page 6: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/6.jpg)
AlgorithmicIntelligenceLab
• Noveltydetection• Givenpre-trained(deep)classifier,• Detectwhetheratestsampleisfromin-distribution(i.e.,trainingdistributionbyclassifier)ornot(e.g.,out-of-distribution/adversarialsamples)
WhatisNoveltyDetection?
6
Decisionboundary
Abnormalsample
![Page 7: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/7.jpg)
AlgorithmicIntelligenceLab
• Noveltydetection• Givenpre-trained(deep)classifier,• Detectwhetheratestsampleisfromin-distribution(i.e.,trainingdistributionbyclassifier)ornot(e.g.,out-of-distribution/adversarialsamples)
• Itcanbeusefulformanymachinelearningproblems:
WhatisNoveltyDetection?
7
Decisionboundary
Abnormalsample
Calibration[Guoetal.,2017]
Ensemblelearning[Leeetal.,2017]
Incrementallearning[Rebuff etal.,2017]
![Page 8: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/8.jpg)
AlgorithmicIntelligenceLab
• Noveltydetection• Givenpre-trained(deep)classifier,• Detectwhetheratestsampleisfromin-distribution(i.e.,trainingdistributionbyclassifier)ornot(e.g.,out-of-distribution/adversarialsamples)
• ItisalsoindispensablewhendeployingDNNsinreal-worldsystems [Amodei etal.,2016]
WhatisNoveltyDetection?
8
Decisionboundary
Abnormalsample
Autonomousdrive Secureauthenticationsystem
![Page 9: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/9.jpg)
AlgorithmicIntelligenceLab
• Howtosolvethisproblem?• Threshold-basedDetector[Hendrycks etal.,2017,Liangetal.,2018]
WhatisNoveltyDetection?
9
[Testsample] [DeepClassifier]
score10
Ifscore>𝜖:In-distribution
Else:out-of-distribution
![Page 10: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/10.jpg)
AlgorithmicIntelligenceLab
• Howtosolvethisproblem?• Threshold-basedDetector[Hendrycks etal.,2017,Liangetal.,2018]
WhatisNoveltyDetection?
10
[Testsample] [DeepClassifier]
score10
Ifscore>𝜖:In-distribution
Else:out-of-distribution
Howtogetconfidencescore
![Page 11: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/11.jpg)
AlgorithmicIntelligenceLab
• Howtosolvethisproblem?• Threshold-basedDetector[Hendrycks etal.,2017,Liangetal.,2018]
• Utilizingaposterior distribution• 1.Maximumvalueorentropyofposterior[Hendrycks etal.,2017]
• 2.Inputandoutputprocessing[Liangetal.,2018]
• 3.Bayesianinference[Lietal.,2017]andensembleofclassifier[Balajietal.,2017]
WhatisNoveltyDetection?
11
[Testsample] [DeepClassifier]
Softmax
Persiancat
tigercat
0.120.18
![Page 12: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/12.jpg)
AlgorithmicIntelligenceLab
• Howtosolvethisproblem?• Threshold-basedDetector[Hendrycks etal.,2017,Liangetal.,2018]
• UtilizingahiddenfeaturesfromDNNs• 1.Localintrinsicdimensionality[Maetal.,2018]
• 2.Mahalanobis distance[Leeetal.,2018b]
WhatisNoveltyDetection?
12
[Testsample] [DeepClassifier]
Softmax
Persiancat
tigercat
0.120.18
![Page 13: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/13.jpg)
AlgorithmicIntelligenceLab
1. Introduction• Whatisnoveltydetection?• Overview
2. UtilizingthePosteriorDistribution• Baselinemethod• Post-processingmethod
3. UtilizingtheHiddenFeatures• Localintrinsicdimensionality• Mahalanobis distance-basedscore
TableofContents
13
![Page 14: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/14.jpg)
AlgorithmicIntelligenceLab
• Remindthatclassificationisfindinganunknownposteriordistribution,i.e.,P(Y|X)
• Howtomodelourposteriordistribution:Softmax classifierwithDNNs
• WhereishiddenfeaturesfromDNNs
UtilizingthePosteriorDistribution
14
Inputspace Outputspace
![Page 15: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/15.jpg)
AlgorithmicIntelligenceLab
• Remindthatclassificationisfindinganunknownposteriordistribution,i.e.,P(Y|X)
• Howtomodelourposteriordistribution:Softmax classifierwithDNNs
• WhereishiddenfeaturesfromDNNs
• Naturalchoiceforconfidencescore• 1.maximumvalueofposteriordistribution
• 2.entropyofposteriordistribution
UtilizingthePosteriorDistribution
15
Inputspace Outputspace
![Page 16: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/16.jpg)
AlgorithmicIntelligenceLab
• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution
UtilizingthePosteriorDistribution
16
[Input] [Deepclassifier]
Ifscore>𝜖:In-distribution
Else:out-of-distribution
![Page 17: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/17.jpg)
AlgorithmicIntelligenceLab
• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution
• Evaluation:detectingout-of-distribution• AssumethatwehaveclassifiertrainedonMNISTdataset• Detectingout-of-distributionforthisclassifier
UtilizingthePosteriorDistribution
17
[Input] [Deepclassifier]
Ifscore>𝜖:In-distribution
Else:out-of-distribution
In-distribution Out-of-distribution
Predictivedist.
Data
![Page 18: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/18.jpg)
AlgorithmicIntelligenceLab
• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution
• Evaluation:detectingout-of-distribution• TP=truepositive/FN=falsenegative/TN=truenegative/FP=falsepositive
UtilizingthePosteriorDistribution
18
[Input] [Deepclassifier]
Ifscore>𝜖:In-distribution
Else:out-of-distribution
• AUROC• AreaunderROCcurve• ROCcurve=relationshipbetweenTPRandFPR
• AUPR(AreaunderthePrecision-Recallcurve)• AreaunderPRcurve• PRcurve=relationshipbetweenprecisionandrecall
![Page 19: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/19.jpg)
AlgorithmicIntelligenceLab
• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution
• Evaluation:detectingout-of-distribution• Imageclassification(computervision)
UtilizingthePosteriorDistribution
19
[Input] [Deepclassifier]
Ifscore>𝜖:In-distribution
Else:out-of-distribution
Baselinemethodisbetterthanrandomdetector
![Page 20: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/20.jpg)
AlgorithmicIntelligenceLab
• Baselinedetector[Hendrycks etal.,2017]• Confidencescore=maximumvalueofpredictivedistribution
• Evaluation:detectingout-of-distribution• Textcategorization(NLP)
• Out-of-distribution• 5Newsgroupsfor15Newsgroups• 2ReutersforReuters6• 12Reutersfor40Reuters
UtilizingthePosteriorDistribution
20
[Input] [Deepclassifier]
Ifscore>𝜖:In-distribution
Else:out-of-distribution
![Page 21: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/21.jpg)
AlgorithmicIntelligenceLab
• ODINdetector[Liangetal.,2018]• Calibratingtheposteriordistributionusingpost-processing
• Twotechniques• Temperaturescaling
• Relaxingtheoverconfidencebysmoothingtheposteriordistribution
UtilizingthePosteriorDistribution
21
Temperaturescalingparameter
![Page 22: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/22.jpg)
AlgorithmicIntelligenceLab
• ODINdetector[Liangetal.,2018]• Calibratingtheposteriordistributionusingpost-processing
• Twotechniques• Temperaturescaling
• Inputpreprocessing
UtilizingthePosteriorDistribution
22
Magnitudeofnoise
![Page 23: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/23.jpg)
AlgorithmicIntelligenceLab
• ODINdetector[Liangetal.,2018]• Calibratingtheposteriordistributionusingpost-processing
• Twotechniques• Temperaturescaling
• Inputpreprocessing
• Usingtwomethods,theauthorsdefineconfidencescoreasfollows:
UtilizingthePosteriorDistribution
23
![Page 24: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/24.jpg)
AlgorithmicIntelligenceLab
• ODINdetector[Liangetal.,2018]• Calibratingtheposteriordistributionusingpost-processing
• Twotechniques• Temperaturescaling
• Inputpreprocessing
• Usingtwomethods,theauthorsdefineconfidencescoreasfollows:
• Howtoselecthyper-parameters• Validation
• 1000imagesfromin-distribution(positive)• 1000imagesfromout-of-distribution(negative)
UtilizingthePosteriorDistribution
24
![Page 25: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/25.jpg)
AlgorithmicIntelligenceLab
• Experimentalresults
UtilizingthePosteriorDistribution
25
![Page 26: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/26.jpg)
AlgorithmicIntelligenceLab
1. Introduction• Whatisnoveltydetection?• Overview
2. UtilizingthePosteriorDistribution• Baseline method• Post-processingmethod
3. UtilizingtheHiddenFeatures• Localintrinsicdimensionality• Mahalanobis distance-basedscore
TableofContents
26
![Page 27: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/27.jpg)
AlgorithmicIntelligenceLab
• Motivation• HiddenfeaturesfromDNNscontainmeaningfulfeaturesfromtrainingdata
• Theycanbeusefulfordetectingabnormalsamples!
UtilizingtheHiddenFeatures
27
LotsofData
Objects
Edge Parts
![Page 28: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/28.jpg)
AlgorithmicIntelligenceLab
• LocalIntrinsicDimensionality(LID)[Maetal.,2018]• Expansiondimension
• Rateofgrowthinthenumberofdataencounteredasthedistancefromthereferencesampleincreases(𝑉 isvolume)
UtilizingtheHiddenFeatures
28
![Page 29: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/29.jpg)
AlgorithmicIntelligenceLab
• LocalIntrinsicDimensionality(LID)[Maetal.,2018]• Expansiondimension
• Rateofgrowthinthenumberofdataencounteredasthedistancefromthereferencesampleincreases(𝑉 isvolume)
• LID=expansiondimensioninthestatisticalsetting
• Where𝐹 isanalogoustothevolumeinequation(1)
UtilizingtheHiddenFeatures
29
![Page 30: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/30.jpg)
AlgorithmicIntelligenceLab
• LocalIntrinsicDimensionality(LID)[Maetal.,2018]• Expansiondimension
• Rateofgrowthinthenumberofdataencounteredasthedistancefromthereferencesampleincreases(𝑉 isvolume)
• LID=expansiondimensioninthestatisticalsetting
• Where𝐹 isanalogoustothevolumeinequation(1)• EstimationofLID[Amsaleg etal.,2015]
UtilizingtheHiddenFeatures
30
distancebetweensampleanditsk-th nearestneighbor
![Page 31: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/31.jpg)
AlgorithmicIntelligenceLab
• MotivationofLID• Abnormalsamplemightbescatteredcomparedtonormalsamples
• ThisimpliesthatLIDcanbeusefulfordetectingabnormalsamples!
UtilizingtheHiddenFeatures
31
![Page 32: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/32.jpg)
AlgorithmicIntelligenceLab
• MotivationofLID• Abnormalsamplemightbescatteredcomparedtonormalsamples
• ThisimpliesthatLIDcanbeusefulfordetectingabnormalsamples!
• Evaluation:detectingadversarialsamples[Szegedy,etal.,2013]• Misclassifiedexamplesthatareonlyslightlydifferentfromoriginalexamples
UtilizingtheHiddenFeatures
32
*Thistopicwillbecoveredinthenextlecture
![Page 33: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/33.jpg)
AlgorithmicIntelligenceLab
• MotivationofLID• Abnormalsamplemightbescatteredcomparedtonormalsamples
• ThisimpliesthatLIDcanbeusefulfordetectingabnormalsamples!
• Evaluation:detectingadversarialsamples[Szegedy,etal.,2013]
UtilizingtheHiddenFeatures
33
=
![Page 34: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/34.jpg)
AlgorithmicIntelligenceLab
• Empiricaljustification
• Adversarialsamples(generatedbyOPTattack[Carlini etal.,2017])canbedistinguishedusingLID
• LIDsfromlow-levellayersarealsousefulindetection
UtilizingtheHiddenFeatures
34
![Page 35: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/35.jpg)
AlgorithmicIntelligenceLab
• Mainresultsondetectingadversarialattacks• Testedmethod
• Bayesianuncertainty(BU)andDensityestimator(DE)[Feinman etal.,2017]
• LIDoutperformsallbaselinemethods
UtilizingtheHiddenFeatures
35
![Page 36: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/36.jpg)
AlgorithmicIntelligenceLab
• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]
UtilizingtheHiddenFeatures
36
![Page 37: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/37.jpg)
AlgorithmicIntelligenceLab
• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]• Givenpre-trainedSoftmax classifierwithDNNs
UtilizingtheHiddenFeatures
37
![Page 38: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/38.jpg)
AlgorithmicIntelligenceLab
• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]• Givenpre-trainedSoftmax classifierwithDNNs
• Inducingagenerativeclassifieronhiddenfeaturespace
UtilizingtheHiddenFeatures
38
penultimate
![Page 39: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/39.jpg)
AlgorithmicIntelligenceLab
• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]• Givenpre-trainedSoftmax classifierwithDNNs
• Inducingagenerativeclassifieronhiddenfeaturespace
• Motivation:connectionbetween softamx andgenerativeclassifier(LDA)
UtilizingtheHiddenFeatures
39
penultimate
~ =
![Page 40: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/40.jpg)
AlgorithmicIntelligenceLab
• Mahalanobis distance-basedconfidencescore[Leeetal.,2018]• Givenpre-trainedSoftmax classifierwithDNNs
• Inducingagenerativeclassifieronhiddenfeaturespace
• Theparametersofgenerativeclassifier=samplemeansandcovariance• Giventrainingdata
UtilizingtheHiddenFeatures
40
penultimate
![Page 41: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/41.jpg)
AlgorithmicIntelligenceLab
• Usinggenerativeclassifier,wedefinenewconfidencescore:
• Measuringthelogoftheprobabilitydensitiesofthetestsample
UtilizingtheHiddenFeatures
41
![Page 42: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/42.jpg)
AlgorithmicIntelligenceLab
• Usinggenerativeclassifier,wedefinenewconfidencescore:
• Measuringthelogoftheprobabilitydensitiesofthetestsample
• Intuition
UtilizingtheHiddenFeatures
42
![Page 43: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/43.jpg)
AlgorithmicIntelligenceLab
• Usinggenerativeclassifier,wedefinenewconfidencescore:
• Measuringthelogoftheprobabilitydensitiesofthetestsample
• Boostingtheperformance• Inputpre-processing
• MotivatedbyODIN[Liangetal.,2018]
UtilizingtheHiddenFeatures
43
![Page 44: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/44.jpg)
AlgorithmicIntelligenceLab
• Usinggenerativeclassifier,wedefinenewconfidencescore:
• Measuringthelogoftheprobabilitydensitiesofthetestsample
• Boostingtheperformance• Inputpre-processing
• Featureensemble
UtilizingtheHiddenFeatures
44
FittingGaussianusingfeaturesfromintermediatelayers
![Page 45: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/45.jpg)
AlgorithmicIntelligenceLab
• Usinggenerativeclassifier,wedefinenewconfidencescore:
• Measuringthelogoftheprobabilitydensitiesofthetestsample
• Boostingtheperformance• Inputpre-processing
• Featureensemble
• Intuition:low-levelfeaturealsocanbeusefulfordetectingabnormalsamples
UtilizingtheHiddenFeatures
45
FittingGaussianusingfeaturesfromintermediatelayers
![Page 46: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/46.jpg)
AlgorithmicIntelligenceLab
• Mainalgorithm
• Remarkthat• Wecombinetheconfidencescoresfrommultiplelayersusingweightedensemble
• Ensembleweightsareselectedbyutilizingthevalidationset
UtilizingtheHiddenFeatures
46
![Page 47: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/47.jpg)
AlgorithmicIntelligenceLab
• Experimentalresultsondetectingout-of-distribution• Contributionbyeachtechnique
UtilizingtheHiddenFeatures
47
Baseline[13]:maximumvalueofposteriordistributionODIN[21]:maximumvalueofposteriordistributionafterpost-processingOurs:theproposedMahalanobis distance-basedscore
![Page 48: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/48.jpg)
AlgorithmicIntelligenceLab
• Experimentalresultsondetectingout-of-distribution• Mainresults
• Forallcases,oursoutperformsODINandbaselinemethod• Validationconsistsof1Kdatafromeachin- andout-of-distributionpair• Validationconsistsof1Kdatafromeachin- andcorrespondingFGSMdata
• Noinformationaboutout-of-distribution
UtilizingtheHiddenFeatures
48
![Page 49: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/49.jpg)
AlgorithmicIntelligenceLab
• Experimentalresultsondetectingadversarialattacks• Mainresults
• Foralltestedcases,ourmethodoutperformsLIDandKDestimator• Forunseenattacks,ourmethodisstillworkingwell
• FGSMsamplesdenotedby“seen”areusedforvalidation
UtilizingtheHiddenFeatures
49
![Page 50: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/50.jpg)
AlgorithmicIntelligenceLab
• Inthislecture,wecovervariousmethodsfordetectingabnormalsampleslikeout-of-distributionandadversarialsamples• Posteriordistribution-basedmethods• Hiddenfeature-basedmethods
• Therearealsotrainingmethodsforobtainingmorecalibratedscores• Ensembleofclassifier[Balajietal.,2017]• Bayesiandeepmodels[Lietal.,2017]• CalibrationlosswithGAN[Leeetal.,2018a]
• Suchmethodscanbeusefulformanymachinelearningapplications• Activelearning[Galetal.,2017]• Incrementallearning[Rebuff etal.,2017]• Ensemblelearning[Leeetal.,2017]• Networkcalibration[Guoetal.,2017]
Summary
50
![Page 51: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/51.jpg)
AlgorithmicIntelligenceLab
[Hendrycks etal.,2017]Abaselinefordetectingmisclassifiedandout-of-distributionexamplesinneuralnetworks. InICLR2017.https://arxiv.org/abs/1610.02136
[Maetal.,2018]CharacterizingAdversarialSubspacesUsingLocalIntrinsicDimensionality. InICLR,2018.https://openreview.net/pdf?id=B1gJ1L2aW
[Feinman etal.,2017]Detectingadversarialsamplesfromartifacts. arXiv preprintarXiv:1703.00410,2017.https://arxiv.org/abs/1703.00410
[Lee,etal.,2018a]TrainingConfidence-calibratedClassifiersforDetectingOut-of-DistributionSamples,InICLR,2018.https://arxiv.org/abs/1711.09325
[Lee,etal.,2018b]ASimpleUnifiedFrameworkforDetectingOut-of-DistributionSamplesandAdversarialAttacks,InNIPS,2018.https://arxiv.org/abs/1807.03888
[Liang,etal.,2018]PrincipledDetectionofOut-of-DistributionExamplesinNeuralNetworks. InICLR,2018.https://arxiv.org/abs/1706.02690
[Goodfellow etal.,2015]Explainingandharnessingadversarialexamples. InICLR,2015.https://arxiv.org/pdf/1412.6572.pdf
[Amodei,etal.,2016]Concreteproblemsinai safety.arXiv preprintarXiv:1606.06565,2016.https://arxiv.org/abs/1606.06565
[Guoetal.,2017]OnCalibrationofModernNeuralNetworks. InICML,2017.https://arxiv.org/abs/1706.04599
[Leeetal.,2017]ConfidentMultipleChoiceLearning.InICML,2017.https://arxiv.org/abs/1706.03475
[Balajietal.,2017]SimpleandScalablePredictiveUncertaintyEstimationusingDeepEnsembles,InNIPS,2017.https://arxiv.org/pdf/1612.01474.pdf
References
51
![Page 52: Novelty Detection for Deep Classifiersalinlab.kaist.ac.kr/resource/Lec13_novelty_detection.pdf · 2020. 4. 25. · •Threshold-based Detector [Hendryckset al., 2017, Liang et al.,](https://reader036.fdocuments.in/reader036/viewer/2022062606/5fe36ede5e56b657b96a030e/html5/thumbnails/52.jpg)
AlgorithmicIntelligenceLab
[Rebuff etal.,2017]iCaRL:IncrementalClassifierandRepresentationLearning. InCVPR,2017.https://arxiv.org/pdf/1611.07725.pdf
[Huangetal.,2017]Denselyconnectedconvolutionalnetworks,InCVPR,2017.https://arxiv.org/abs/1608.06993
[Zagoruyko etal.,2016]Wideresidualnetworks,InBMVC2016.https://arxiv.org/pdf/1605.07146.pdf
[Amsaleg etal.,2015]Estimatinglocalintrinsicdimensionality.InSIGKDD,2015.http://mistis.inrialpes.fr/~girard/Fichiers/p29-amsaleg.pdf
[Szegedy etal.,2013]Intriguingpropertiesofneuralnetworks. arXiv preprintarXiv:1312.6199,2013.https://arxiv.org/abs/1312.6199
[Lietal.,2017]DropoutInferenceinBayesianNeuralNetworkswithAlpha-divergences,InICML,2017.https://arxiv.org/abs/1703.02914
[Galetal.,2017]DeepBayesianActiveLearningwithImageData,InICML,2017.https://arxiv.org/abs/1703.02910
[Carlini etal.,2017]Towardsevaluatingtherobustnessofneuralnetworks.In IEEESP,2017.https://arxiv.org/abs/1608.04644
References
52