U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation...
Transcript of U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation...
![Page 1: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/1.jpg)
U-Net: Convolutional
Networks for Biomedical
Image SegmentationIng. Milan NΔmΓ½
PhD candidate
FEL ΔVUT, 7. 12. 2018
![Page 2: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/2.jpg)
Outline
1. Artificial neural networks
2. Convolutional neural networks (classification)
3. Convolutional neural networks (segmentation)
1. Sliding-window setup
2. U-Net
![Page 3: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/3.jpg)
Artificial neural networks
Perceptron model
π¦ π₯ = π πππ π€. π₯ + π
Artificial Neural Network
β’ Hidden layers
β’ Non-linear activation
function (tanh, sigmoid)
β’ Cost function
β’ Back-propagation algorithm
![Page 4: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/4.jpg)
Artificial neural networks
Artificial Neural Network
β’ Hidden layers
β’ Non-linear activation
function (tanh, sigmoid)
β’ Cost function
β’ Back-propagation algorithm
Multi-class classification
Activation function β softmax
ππ =exp(π₯π)
Οπ exp(π₯π)
Cost function β cross entropy
πΆ = β
π
ππlog(ππ)
ππ - target probability for output unit j
ππ - probability output for j
![Page 5: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/5.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012)
Winner of ImageNet Large Scale Visual
Recognition Challenge (ILSVRC) 2012
1.2 M high-resolution training images
50k validation images
150k testing images
1000 classes
![Page 6: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/6.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012) - AlexNet
Winner of ImageNet Large Scale Visual
Recognition Challenge (ILSVRC) 2012
1.2 M high-resolution training images
50k validation images
150k testing images
1000 classes
![Page 7: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/7.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012)
Convolution
= application of a filter
![Page 8: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/8.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012)
β’ Rectified Linear Units (ReLUs)
β’ Non-saturating nonlinearity
β’ π π₯ = max(0, π₯)
β’ Traininig on multiple GPUs
β’ 5 conv layers
β’ 3 fully-connected
β’ output β 1000-way softmax
![Page 9: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/9.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012)
60 M parameters
But overfitting
Data augmentation
Dropout β Learning Less to Learn Better
![Page 10: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/10.jpg)
Convolutional Neural Networks (CNN)
(Krizhevsky et al., 2012)
60 M parameters
But overfitting
Data augmentation
Dropout β Learning Less to Learn Better
![Page 11: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/11.jpg)
![Page 12: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/12.jpg)
Convolutional Neural Networks (CNN) β
AlexNet performance
Canziani, A., Paszke, A., & Culurciello, E. (2016). An analysis of deep neural network
models for practical applications. arXiv preprint arXiv:1605.07678.
![Page 13: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/13.jpg)
Convolutional Neural Networks -
Segmentation
(Ciresan et al., 2012)
Neuronal structures
Electron microscopy (EM) images
Segment neuron membranes
CNN as pixel classifier
ISBI 2012 EM Segmentation Challenge
![Page 14: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/14.jpg)
Convolutional Neural Networks -
Segmentation
β’ 1-4 stages of conv and max-pooling layes
β’ Several fully connected layers
β’ Softmax activation function
![Page 15: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/15.jpg)
Convolutional Neural Networks -
Segmentation
30 images at 512x512
~50k membrane pixels per image
~50k non-memberane pixels per images
=> 3 M training examples
+ data augmentation (mirroring and rotating)
Foveation
Nonuniform sampling
![Page 16: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/16.jpg)
β’ Core i7 3.06 GHz, 24 GB RAM and
four GTX 580 graphic cards
β’ GPU acceleration by a factor of 50
β’ One epoch
β’ N1 β 170 min
β’ N4 β 340 min
β’ 30 epochs => several days
![Page 17: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/17.jpg)
Convolutional Neural Networks β
Segmentation (Sliding-window setup)
Drawbacks
1. Separate runs for each patch +
overlapping patches => slow
2. Localization accuracy vs. the use
of context
![Page 18: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/18.jpg)
U-net architecture
![Page 19: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/19.jpg)
U-net architecture
Data augmentation
Separation of touching objects
![Page 20: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/20.jpg)
U-net architecture
Data augmentation
Separation of touching objects π€ π₯ = π€π π₯ + π€0 β exp(β(π1(π₯) + π2(π₯))
2
2π2)
Balances class
frequencies
Distance to
the border of
the nearest
cell
Distance to
the border of
the second
nearest cell
![Page 21: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/21.jpg)
U-net architecture
Data augmentation
Separation of touching objects π€ π₯ = π€π π₯ + π€0 β exp(β(π1(π₯) + π2(π₯))
2
2π2)
Balances class
frequencies
Distance to
the border of
the nearest
cell
Distance to
the border of
the second
nearest cell
πΈ =
π₯βΞ©
π€ π₯ log(ππ π₯ (π₯))Cross entropy loss
function
ππ π₯ = exp ππ π₯ / πβ²=1
πΎ
exp(ππβ²(π₯))Soft-max activation
function
![Page 22: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/22.jpg)
U-net
β’ Training time: 10 h
Sliding-window
technique
![Page 23: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/23.jpg)
Demonstration - Finding and Measuring Lungs
in CT Data
?
Notebook:
https://www.kaggle.com/tore
gil/a-lung-u-net-in-
keras/notebook
![Page 24: U-Net: Convolutional Networks for Biomedical Image ...Β Β· U-net architecture Data augmentation Separation of touching objects = π + 0βexp(β (π1( )+π2( ))2 2π2) Balances](https://reader035.fdocuments.in/reader035/viewer/2022062915/5e970c55b8fc82548836749b/html5/thumbnails/24.jpg)
References
[1] Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net:
Convolutional networks for biomedical image segmentation. In International
Conference on Medical image computing and computer-assisted intervention
(pp. 234-241). Springer, Cham.
[2] Ciresan, D., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2012).
Deep neural networks segment neuronal membranes in electron microscopy
images. In Advances in neural information processing systems (pp. 2843-
2851).
[3] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet
classification with deep convolutional neural networks. In Advances in neural
information processing systems (pp. 1097-1105).