AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)
-
Upload
amazon-web-services -
Category
Technology
-
view
260 -
download
0
Transcript of AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)
![Page 1: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Advisors: Robert Chang, Jeff Ullman, Andreas Paepcke
November 30, 2016
Automatic Grading of Diabetic
Retinopathy through Deep LearningApaar Sadhwani, Leo Tam, and Jason Su
MAC403
![Page 2: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/2.jpg)
Problem, Data and Motivation Motivation:
Affects ~100M, many in developed, ~45% of diabetics Make process faster, assist ophthalmologist, self-help Widespread disease, enable early diagnosis/care
Given fundus image Rate severity of Diabetic Retinopathy 5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe) Hard classification (may solve as ordinal though) Metric: quadratic weighted kappa, (pred – real)2 penalty
Data from Kaggle (California Healthcare Foundation, EyePACS) ~35,000 training images, ~54,000 test images High resolution: variable, more than 2560 x 1920
![Page 3: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/3.jpg)
Problem, Data and Motivation Motivation:
Affects ~100M, many in developed, ~45% of diabetics Make process faster, assist ophthalmologist, self-help Widespread disease, enable early diagnosis/care
Given fundus image Rate severity of Diabetic Retinopathy 5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe) Hard classification (may solve as ordinal though) Metric: quadratic weighted kappa, (pred – real)2 penalty
Data from Kaggle (California Healthcare Foundation, EyePACS) ~35,000 training images, ~54,000 test images High resolution: variable, more than 2560 x 1920
![Page 4: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/4.jpg)
Example images
Class 0 (normal) Class 4 (severe)
![Page 5: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/5.jpg)
Problem, Data and Motivation Motivation:
Affects ~100M, many in developed, ~45% of diabetics Make process faster, assist ophthalmologist, self-help Widespread disease, enable early diagnosis/care
Given fundus image Rate severity of Diabetic Retinopathy 5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe) Hard classification (may solve as ordinal though) Metric: quadratic weighted kappa, (pred – real)2 penalty
Data from Kaggle (California Healthcare Foundation, EyePACS) ~35,000 training images, ~54,000 test images High resolution: variable, more than 2560 x 1920
![Page 6: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/6.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
Image size Batch Size224 x 224 1282K x 2K 2
![Page 7: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/7.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
Class 0 1
2 3
4
![Page 8: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/8.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples Class 2
![Page 9: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/9.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
![Page 10: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/10.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Mentioned in problem statement- Confirmed with doctors
![Page 11: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/11.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
![Page 12: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/12.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Hard classification non-differentiable- Backprop difficult
0 1Truth
2 3 4
Penalty/Loss
Class
![Page 13: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/13.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Hard classification non-differentiable- Backprop difficult
0 1Truth
2 3 4
Predict1
Penalty/Loss
Class
![Page 14: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/14.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Hard classification non-differentiable- Backprop difficult
0 1Truth
2 3 4
Predict2
Penalty/Loss
Class
![Page 15: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/15.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Hard classification non-differentiable- Backprop difficult
0 1Truth
2 3 4
Predict3
Penalty/Loss
Class
![Page 16: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/16.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Hard classification non-differentiable- Backprop difficult
0 1Truth
2 3 4
Penalty/Loss
Class
![Page 17: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/17.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Squared error approximation?- Differentiable
0 1Truth
2 3 4
Penalty/Loss
Class2.5
![Page 18: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/18.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Naïve: 3 class problem, or all zeros!- Learn all classes separately: 1 vs All?- Balanced while training
- At test time?
![Page 19: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/19.jpg)
Challenges High resolution images
Atypical in vision, GPU batch size issues
Discriminative features small Grading criteria:
not clear (EyePACS guidelines) learn from data
Incorrect labeling Artifacts in ~40% images Optimizing approach to QWK Severe class imbalance
class 0 dominates
Too few training examples
- Big learning models take more data!- Harness test set?
![Page 20: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/20.jpg)
Conventional Approaches Literature survey:
Hand-designed features to pick each component
Clean images, small datasets Optic disk, exudate segmentation: fail
due to artifacts SVM: poor performance
![Page 21: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/21.jpg)
Conventional Approaches Literature survey:
Hand-designed features to pick each component
Clean images, small datasets Optic disk, exudate segmentation: fail
due to artifacts SVM: poor performance
![Page 22: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/22.jpg)
Our Approach
1. Registration, Pre-processing2. Convolutional Neural Nets (CNNs)3. Hybrid Architecture
![Page 23: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/23.jpg)
Step 1: Pre-processing
Registration
Hough circles, remove outside portion
Downsize to common size (224 x 224, 1K x 1K)
Color correction Normalization (mean, variance)
![Page 24: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/24.jpg)
Step 2: CNNs
3 Conv layers (depth 96)
MaxPool (stride2)
3 Conv layers (depth 384)
MaxPool (stride2)
3 Conv layers (depth 1024)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
3 Conv layers (depth 256)
MaxPool (stride2)
Network in Network architecture 7.5M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
![Page 25: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/25.jpg)
Step 2: CNNs
3 Conv layers (depth 96)
MaxPool (stride2)
3 Conv layers (depth 384)
MaxPool (stride2)
3 Conv layers (depth 1024)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
3 Conv layers (depth 256)
MaxPool (stride2)
Network in Network architecture 7.5M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
![Page 26: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/26.jpg)
Step 2: CNNs
3 Conv layers (depth 96)
MaxPool (stride2)
3 Conv layers (depth 384)
MaxPool (stride2)
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
3 Conv layers (depth 256)
MaxPool (stride2)
Network in Network architecture 2.2M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
![Page 27: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/27.jpg)
Step 2: CNNs
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Network in Network architecture 2.2M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
![Page 28: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/28.jpg)
Step 2: CNNs
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Network in Network architecture 2.2M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
![Page 29: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/29.jpg)
Step 2: CNNs
Network in Network architecture 2.2M parameters No FC layers, spatial average pooling instead
Transfer learning (ImageNet) Variable learning rates
Low for “ImageNet” layers Schedule
Combat lack of data, over-fitting Dropout, Early stopping Data augmentation (flips, rotation)
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
![Page 30: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/30.jpg)
Step 2: CNN Experiments
What image size to use? Strategize using 224 x 224 -> extend to 1024 x 1024
What loss function? Mean squared error (MSE) Negative Log Likelihood (NLL) Linear Combination (annealing)
Class imbalance Even sampling -> true sampling
![Page 31: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/31.jpg)
Step 2: CNN Experiments
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Nolearning
Loss Function Sampling Result
Image size: 224 x 224
![Page 32: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/32.jpg)
Step 2: CNN Experiments
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Nolearning
Loss Function Sampling Result
MSE Fails to learn
Image size: 224 x 224
![Page 33: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/33.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
MSE Fails to learn
MSE Fails to learn
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Nolearning
![Page 34: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/34.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
MSE Fails to learn
MSE Fails to learn
NLL Kappa < 0.1
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Nolearning
![Page 35: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/35.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
MSE Fails to learn
MSE Fails to learn
NLL Kappa < 0.1
NLL Kappa = 0.29
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
Nolearning
![Page 36: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/36.jpg)
Step 2: CNN Experiments
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
0.01x step size
Loss Function Sampling Result
NLL(top layers only)
Kappa = 0.29
Image size: 224 x 224
![Page 37: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/37.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
NLL(top layers only)
Kappa = 0.29
NLL Kappa = 0.42
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
0.01x step size
![Page 38: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/38.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
NLL(top layers only)
Kappa = 0.29
NLL Kappa = 0.42
NLL Kappa = 0.51
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
0.01x step size
![Page 39: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/39.jpg)
Step 2: CNN Experiments
Loss Function Sampling Result
NLL(top layers only)
Kappa = 0.29
NLL Kappa = 0.42
NLL Kappa = 0.51
MSE Kappa = 0.56
Image size: 224 x 224
3 Conv layers (depth 384, 64, 5)
MaxPool (stride2)
AvgPool
Input Image
Class probabilities
0.01x step size
![Page 40: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/40.jpg)
Step 2: CNN Results
![Page 41: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/41.jpg)
Step 2: CNN Results
![Page 42: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/42.jpg)
Computing Setup
Amazon EC2: GPU nodes, VPC, Amazon EBS-optimized Single GPU nodes for 224 x 224 (g2.2xlarge) Multi-GPU nodes for 1K x 1K (g2.8xlarge)
EBS, Amazon S3
Used Python for processing
Torch library (Lua) for training
![Page 43: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/43.jpg)
Computing Setup
Data EBS (gp2)
Model Expt.
1 or 4 GPU node on EC2
![Page 44: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/44.jpg)
Computing Setup
Data 1 Data 2EBS (gp2) EBS (gp2)
Snapshot (S3)
Model Expt.
GPU node on EC2
![Page 45: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/45.jpg)
Computing Setup
Master
Data 1 Data 2Central Node
Model 2
Model 1
Model 10
EBS (gp2)
…
EBS-optimized
EBS (gp2)
Snapshot (S3)
VPC on EC2
Model Expt.
GPU node on EC2
![Page 46: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/46.jpg)
Computing Setup
Master
Data 1 Data 2Central Node
Model 2
Model 1
Model 10
EBS (gp2)
…
EBS-optimized
EBS (gp2)
Snapshot (S3)
VPC on EC2
Model Expt.
GPU node on EC2~200 MB/s
![Page 47: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/47.jpg)
Computing Setup
Master 2
Data 1 Data 2Central Node
Model 12
Model 11
Model 20
EBS (gp2)
…
EBS-optimized
EBS (gp2)
Snapshot (S3)
VPC on EC2
Master 1
Central Node
Model 2
Model 1
Model 10…
EBS-optimized VPC on EC2
![Page 48: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/48.jpg)
Computing Setup
g2.2xlarge1 GPU node on EC2
4 GB GPU memoryBatch size: 128 images of 224 x 224
![Page 49: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/49.jpg)
Computing Setup
g2.2xlarge1 GPU node on EC2
4 GB GPU memoryBatch size: 128 images of 224 x 224
!! Batch size: 8 images of 1024 x 1024 !!
![Page 50: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/50.jpg)
Computing Setup
g2.2xlarge1 GPU node on EC2
4 GB GPU memoryBatch size: 128 images of 224 x 224
!! Batch size: 8 images of 1024 x 1024 !!
g2.8xlarge4 GPU node on EC2
16 GB GPU memoryData ParallelismBatch size: ~28 images of 1024 x 1024
![Page 51: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/51.jpg)
Step 3: Hybrid Architecture
2048 1024
64 tiles of256 x 256
MainNetwork
Fuse
Class probabilities
LesionDetector
![Page 52: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/52.jpg)
Lesion Detector
Web viewer and annotation tool Lesion annotation Extract image patches Train lesion classifier
![Page 53: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/53.jpg)
Viewer and Lesion Annotation
![Page 54: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/54.jpg)
Viewer and Lesion Annotation
![Page 55: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/55.jpg)
Lesion Annotation
![Page 56: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/56.jpg)
Extracted Image Patches
![Page 57: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/57.jpg)
Train Lesion Detector
Only hemorrhages so far Positives: 1866 extracted patches from 216
images/subjects Negatives: ~25k class-0 images Pre-processing/augmentation
Crop random 256 x 256 image from input, flips Pre-trained Network in Network architecture Accuracy: 99% for Negatives, 76% for Positives
![Page 58: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/58.jpg)
Train Lesion Detector
Only hemorrhages so far Positives: 1866 extracted patches from 216
images/subjects Negatives: ~25k class-0 images Pre-processing/augmentation
Crop random 256 x 256 image from input, flips Pre-trained Network in Network architecture Accuracy: 99% for Negatives, 76% for Positives
![Page 59: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/59.jpg)
Train Lesion Detector
Only hemorrhages so far Positives: 1866 extracted patches from 216
images/subjects Negatives: ~25k class-0 images Pre-processing/augmentation
Crop random 256 x 256 image from input, flips Pre-trained Network in Network architecture Accuracy: 99% for Negatives, 76% for Positives
![Page 60: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/60.jpg)
Train Lesion Detector
Only hemorrhages so far Positives: 1866 extracted patches from 216
images/subjects Negatives: ~25k class-0 images Pre-processing/augmentation
Crop random 256 x 256 image from input, flips Pre-trained Network in Network architecture Accuracy: 99% for Negatives, 76% for Positives
![Page 61: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/61.jpg)
Train Lesion Detector
Only hemorrhages so far Positives: 1866 extracted patches from 216
images/subjects Negatives: ~25k class-0 images Pre-processing/augmentation
Crop random 256 x 256 image from input, flips Pre-trained Network in Network architecture Accuracy: 99% for Negatives, 76% for Positives
![Page 62: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/62.jpg)
Hybrid Architecture
64 tiles of256 x 256
2048 1024
MainNetwork
Fuse
Class probabilities
LesionDetector
![Page 63: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/63.jpg)
Hybrid Architecture
64 tiles of256 x 256
64 x 31 x 312 x 31 x 31
66 x 31 x 31
2048 1024
2 Conv layers
MainNetwork
Fuse
Class probabilities
LesionDetector
![Page 64: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/64.jpg)
Hybrid Architecture
64 tiles of256 x 256
64 x 31 x 312 x 31 x 31
66 x 31 x 31
2048 1024
2 Conv layers
MainNetwork
Fuse
Class probabilities
LesionDetector
2 x 56 x56
![Page 65: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/65.jpg)
Training Hybrid Architecture
![Page 66: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/66.jpg)
Class probabilities
Training Hybrid Architecture
64 tiles of256 x 256
2048 1024
MainNetwork
Fuse
LesionDetector
![Page 67: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/67.jpg)
Training Hybrid Architecture
64 tiles of256 x 256
Backprop
2048 1024
MainNetwork
Fuse
Class probabilities
LesionDetector
![Page 68: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/68.jpg)
Training Hybrid Architecture
64 tiles of256 x 256
Backprop
2048 1024
MainNetwork
Fuse
Class probabilities
LesionDetector
![Page 69: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/69.jpg)
Other Insights
Supervised-unsupervised learning Distillation Hard-negative mining Other lesion detectors Attention CNNs Both eyes Ensemble
![Page 70: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/70.jpg)
Clinical Importance
3 class problem True “4” problem Combining imaging modalities (OCT) Longitudinal analysis
![Page 71: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/71.jpg)
Many thanks to…
Amazon Web Services AWS Educate AWS Cloud Credits for Research
Robert Chang Jeff Ullman Andreas Paepcke
![Page 72: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/72.jpg)
Thank you!
![Page 73: AWS re:Invent 2016: Automatic Grading of Diabetic Retinopathy through Deep Learning (MAC403)](https://reader031.fdocuments.in/reader031/viewer/2022020411/58714f0d1a28ab55588b7631/html5/thumbnails/73.jpg)
Remember to complete your evaluations!