Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. ·...
Transcript of Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. ·...
![Page 1: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/1.jpg)
Explaining deep learning for identifying structures
and biases in computer vision
A Talk at: Interpretable ML in Vision@ICCV 2019.
Joint work with W. Samek, S. Lapuschkin (nee Bach), G. Montavon,K.-R. Muller, and deserving others
Alexander Binder
October 28, 2019
![Page 2: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/2.jpg)
What is a possible explanation of a prediction?
for images: (Densenet121, Keras+innvestigate, 2019)
• case of images: compute a score for every pixel
image gradient LRP-↵-�
![Page 3: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/3.jpg)
What is a possible explanation of a prediction?
• case of images: compute a score for every pixel• patch-wise classification: label = 1 if patch contains breast
cancer• pixel-wise explanation
• general case: score for every dim of an input samplex = (x1, . . . , xd , . . . , xD)
![Page 4: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/4.jpg)
What is LRP as explanation?
(Densenet121, Keras+innvestigate, 2019)
• given: A. trained model f , B. a prediction f (x) for inputx = (x1, . . . , xd , . . . , xD).
• general case: LRP computes a relevance score rd(x) for everyinput dimension xd of input x explaining the prediction f (x),such that approximately:
f (x) ⇡DX
d=1
rd(x) decomposition with constraints (1)
image gradient LRP-↵-�
![Page 5: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/5.jpg)
What is a possible explanation of a prediction?
![Page 6: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/6.jpg)
What is a possible explanation of a prediction?
![Page 7: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/7.jpg)
What is a possible explanation of a prediction?
![Page 8: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/8.jpg)
Trivial rules
Given f (x), can obtain desired decomposition
f (x) =DX
d=1
rd(x) by e.g. (2)
rd(x) = f (x)/D (3)
rd(x) =
(f (x) d = 1
0 else(4)
• underdetermined, many non-plausible decompositions
• need additional constraints
• theoretical foundation yielding constraints: Deep Taylor framework
• Taylor decomposition of every single neuron with customizedroot points.
![Page 9: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/9.jpg)
Deep Taylor Decomposition
![Page 10: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/10.jpg)
Relevance distribution for one neuron: example ✏-rule
✏-rule:
Ri k(x) / Rkh(wixi ) (5)
Ri k(x)= RkwixiP
i 0 wi 0xi 0 + b + ✏ · sign (6)
• ✏ – dampening factor, numerical stabilization
• recommended for fully connected layers and good for LSTMs(cf. Leila Arras et al.)
• NOT recommended for conv layers
![Page 11: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/11.jpg)
Relevance distribution for one neuron: example ↵-�-rule
�-rule:
Ri k(x) / Rkh(wixi ) (7)
Ri k(x)= Rk
✓(1 + �)
(wixi )+Pi 0(wi 0xi 0)+ + b+
� �(wixi )�P
i 0(wi 0xi 0)� + b�
◆(8)
• � – controls ratio of negative to positive evidence.
• � = 0 only positive evidence (analogous to e.g. guided backprop)
• suitable for conv layers (with modifications: batchnorm layers)
![Page 12: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/12.jpg)
Gradient ⇥ Input?
![Page 13: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/13.jpg)
Gradient ⇥ Input?
![Page 14: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/14.jpg)
Gradient ⇥ Input?
![Page 15: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/15.jpg)
Gradient ⇥ Input?
![Page 16: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/16.jpg)
Gradient ⇥ Input?
![Page 17: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/17.jpg)
Examples (Densenet121, Keras, 2019)
img gradient ↵-�
hybrid rule: � = 0 for conv layers, ✏ = 0.01 for fc layer
![Page 18: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/18.jpg)
Tell them something interesting!
![Page 19: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/19.jpg)
LRP Applied to Variety of Models
![Page 20: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/20.jpg)
LRP Applied to Variety of Tasks
![Page 21: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/21.jpg)
The value of explanations
A. application case: identify action strategies in reinforcementlearning predictors
B. general: Identify Biases in Train+Test data (where labels donot help you at all)
C. medical imaging: Identify Fail Cases without labelling e↵orts�! Iterative Dataset Design
D. application case: LRP in neuroscience
![Page 22: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/22.jpg)
LRP: DNN and Atari Breakout
A. application case: identify action strategies in reinforcementlearning predictors
Trained a reinforcement learning classifier according to Mnih et al’sNature 2016 paper:Volodymyr Mnih et al. Human-level control through deepreinforcement learning,Nature 518, pages 529533, 2015
Conv netimage set
2-n actions(classi er)
RL objective for training:maximize expected future reward
![Page 23: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/23.jpg)
LRP: DNN and Atari Breakout
Trained a reinforcement learning classifier according to Mnih et al’s Nature2016 paper.
Explain a test game. LRP helps to discover strategies: building a tunnel.
Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 24: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/24.jpg)
LRP: DNN and Atari Breakout
Trained a reinforcement learning classifier according to Mnih et al’s Nature2016 paper.
LRP can help to discover strategies: building a tunnel - evolution of focus
during training
epoch 0 and 6Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 25: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/25.jpg)
LRP: DNN and Atari Breakout
Trained a reinforcement learning classifier according to Mnih et al’s Nature2016 paper.
LRP can help to discover strategies: building a tunnel - evolution during
training
epoch 50 and 100Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 26: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/26.jpg)
LRP: DNN and Atari Breakout
LRP can help to find parameters for fast learning of known strategies. Here:
impact of M = replay memory size
Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 27: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/27.jpg)
LRP in reinforcement learning
Interpretability methods (here: LRP) can uncover complexrelationships
Atari Pinball:
move ball 4 times over switch to activate a score multiplier.
.. if there are any
Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 28: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/28.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
C. general: Identify Biases in Train+Test data (where labels donot help you at all)
At first: general images ... less careful about biases
![Page 29: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/29.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
Analyzing Classifiers: Fisher Vectors and Deep Neural Networks, Lapuschkin et
al., CVPR 2016
![Page 30: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/30.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
Image Fisher Vector Deep Neural Net
Analyzing Classifiers: Fisher Vectors and Deep Neural Networks, Lapuschkin et
al., CVPR 2016
![Page 31: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/31.jpg)
SpRAy: semi-automatic discovery of correlations
Lapuschkin et al. Nature Communications 2019:
Principle
• compute heatmaps, pool them into a uniform low resolution 20⇥ 20
• compute binarized similarity wij between heatmaps of samples i and jusing k = log sample size
wij =n1 if i is among the k-nearest neighbors of j (9)
• symmetrize W = (wij)i,j 7! max(wij ,wji )
• compute eigenvalue/vectors of Laplacian L = I � D�1/2WD�1/2
• inspect eigenvalue gaps
![Page 32: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/32.jpg)
SpRAy: DNN and Pascal VOC Aeroplane class
SpRAy: Two Large gaps in low eigenvalues for aeroplane –conspicuous.
Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 33: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/33.jpg)
SpRAy: DNN and Pascal VOC Aeroplane class
• t-sne shows one cluster where aeroplanes have strong evidence on edgesdue to data preparation artefact combined with frequency of blue sky.
• Did not wanted to use center crops: avoid cutting o↵ object parts. Soedges were padded with border pixels. This is used in one part of theaeroplane images as cue.
![Page 34: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/34.jpg)
SpRAy: DNN and Pascal VOC Aeroplane class
Confirm that paddings are a cue:
• images with aeroplane predicted: changing borders to random noisedestroys aeroplane scores
• images with no aeroplane predicted: changing borders to sky blue colorimproved aeroplane score, even random but constant color helps.
![Page 35: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/35.jpg)
SpRAy: DNN and Pascal VOC Aeroplane class
Confirm that paddings are a cue:
• images with aeroplane predicted: changing borders to random noisedestroys aeroplane scores
• images with no aeroplane predicted: changing borders to sky blue colorimproved aeroplane score, even random but constant color helps.
![Page 36: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/36.jpg)
SpRAy: DNN and Pascal VOC Aeroplane class
Result show:
• identified another bias by inspecting heatmaps – this one is hard to seefor humans: at borders (psychologically suppressed as irrelevant!) plusconstant color in one class
Lapuschkin et al., Unmasking Clever Hans predictors and assessing what machines really learn,
Nature Communications, 2019
![Page 37: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/37.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
C. general: Identify Biases in Train+Test data (where labels donot help you at all)
![Page 38: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/38.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
C. general: Identify Biases in Train+Test data (where labels donot help you at all)
and now to something morerelevant please!
Medical datasets
![Page 39: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/39.jpg)
Identify Biases in Train+Test data (where labels do not
help you at all)
Haegele et al., Resolving challenges in deep learning-based analysesof histopathological images using explanation methods, arxiv 2019:
?– Are heatmaps of patch-level classifiers quantifiably meaningfulin terms of resolution at cell nucleus level ? Do they considernuclei as evidence? How good are heatmaps in terms ofmeasured localization accuracy?
?– Are heatmaps useful to resolve biases in histopathology?• systematic biases• class-correlation biases• sampling biases• LRP for evaluating the impact of class sampling ratios
![Page 40: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/40.jpg)
Quantifying heatmaps on cell level
Three datasets: Annotate nuclei densely.
BRCA
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 41: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/41.jpg)
Quantifying heatmaps on cell level
Three datasets: Annotate nuclei densely.
LUAD (lung)
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 42: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/42.jpg)
Quantifying heatmaps on cell level
Three datasets: Annotate nuclei densely.
SKCM (Melanoma)
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 43: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/43.jpg)
Quantifying heatmaps on cell level
Train patch classifier, compute heatmaps.
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 44: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/44.jpg)
Quantifying heatmaps on cell level
Do we need high res methods like LRP or guided BP ? (a lil bitbashing please be forgiven)
LRP GradCAMHaegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 45: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/45.jpg)
Quantifying heatmaps on cell level
Evaluation Data on nucleus level
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 46: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/46.jpg)
Quantifying heatmaps on cell level
Evaluation Data on the level of nuclei:
• Poor sensitivity on mid rangesfor SKCM and BRCA.
• Inspecting heatmaps forSKCM reveals two slides withdense tissue invadinglymphocytes – receivingmoderately positive scores.
• Points at insu�cient samplingof patches with TiLs intraining :) .
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 47: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/47.jpg)
Sampling bias
• left heatmap: false positive scores on unlabeled subclass.
• right heatmap: after augmenting training dataset withnecrosis samples (negative labeled)
training
image
test
image
heatmap
with bias
heatmap
w/o bias
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 48: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/48.jpg)
Sampling bias
Retraining has statistically visible e↵ect.
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 49: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/49.jpg)
Sampling bias
Retraining has a visually visible e↵ect, too.
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 50: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/50.jpg)
Sampling bias
Here: without necrosis samples.
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 51: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/51.jpg)
Sampling bias
Here: with necrosis samples.
your version1 labels and test set error cannot discover it
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 52: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/52.jpg)
Class-correlation bias
training
image
test
image
heatmap
with bias
heatmap
w/o bias
• biases are identifiable
• test set labels are of no help (!) for discovery
• debiasing improves explanations
Haegele et al., Resolving challenges in deep learning-based analyses of histopathological images using explanation
methods, arxiv 2019
![Page 53: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/53.jpg)
Identify Fail Cases without labelling e↵orts:Evaluate Impact of data augmentation
Image scaling ?
orig 80%
100% 66%
![Page 54: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/54.jpg)
Medical Data: Identify Fail Cases without labelling e↵orts
C. medical imaging: Identify Fail Cases without labelling e↵orts�! Iterative Dataset Design
Why not just using test error ?
• some problems: labels very costly, unlabeled data abundant
![Page 55: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/55.jpg)
Identify Fail Cases without labelling e↵orts
More Importantly:
• decide what unlabeled data to add into next iteration of trainand test set
• Interpretability for e�ciency in the selection step beforelabelling!
![Page 56: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/56.jpg)
Identify Fail Cases without labelling e↵orts
More Importantly:
• decide what unlabeled data to add into next iteration of trainand test set – precursor to labelling.
• Interpretability for e�ciency in the selection step beforelabelling!
![Page 57: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/57.jpg)
LRP in Neuroscience
Thomas et al.Analyzing Neuroimaging Data Through Recurrent Deep LearningModels, arxiv 2019
![Page 58: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/58.jpg)
LRP in Neuroscience
Thomas et al.
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models, arxiv 2019
![Page 59: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/59.jpg)
LRP in Neuroscience
Thomas et al.
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models, arxiv 2019
![Page 60: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/60.jpg)
LRP in Neuroscience
Thomas et al.
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models, arxiv 2019
![Page 61: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/61.jpg)
References
![Page 62: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/62.jpg)
References
![Page 63: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/63.jpg)
References
![Page 64: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/64.jpg)
References
![Page 65: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/65.jpg)
New book out
link to the book:
https://www.springer.com/gp/book/
9783030289539
Organization of the book:
• Part I Towards AI Transparency
• Part II Methods for Interpreting AI Systems
• Part III Explaining the Decisions of AI Systems
• Part IV Evaluating Interpretability andExplanations
• Part V Applications of Explainable AI
• 22 Chapters
papers, demos, ice cream at: www.explain-ai.org
![Page 66: Explaining deep learning for identifying structures and biases in … · 2020. 6. 15. · Explaining deep learning for identifying structures and biases in computer vision A Talk](https://reader035.fdocuments.in/reader035/viewer/2022071216/6047b57dccb6eb62d252df26/html5/thumbnails/66.jpg)
Questions?!