GTC EUROPE 2017

AidenceEnhancing Radiology with Artificial Intelligence

Localization in 3D Biomedical Image Datausing Deep Learning

Mark-Jan Harte, CEO

GTC EUROPE 2017

About Aidence

● Founded in 2015, based in Amsterdam

● Deep learning for automatic medical image analysis

● 3rd place in the Kaggle Data Science Bowl 2017

2

GTC EUROPE 2017

Challenges for AI in Radiology

Technical

○ Sample size

○ Class imbalances due to mostly healthy/background tissue present

○ Accurate labeling is a pain

○ Validation dataset for regulatory approval required

3

Common resolution ImageNet: 256x256x3 = 196.608

Common resolution CT scan: 300x512x512x1 = 78.643.200

GTC EUROPE 2017

CT Chest

● Early detection of lung nodules leads to 20% mortality reduction

● Human sensitivity ~80%

Nodules are:

● Small● Anywhere● Highly variable in number

Lung Nodules

4

GTC EUROPE 2017

Detection

Training

● Input is 128x128x7 voxels● Target is 58x58 rectangle mask● Loss: Normalized cross entropy● Output: segmentation (probability map)● Keep fine grained spatial details● Network size not too big● Second network to filter out false positives

○ Larger, less restrictions

Lung Nodules

5

GTC EUROPE 2017

Network architecture

● Fully 3D convolutional○ Efficient inference on big CT scans○ No same padding○ No pooling○ No strides

● Dilated convolutions ○ Reduce resolution more quickly○ Keep network size in check (180K params)

● Weight normalization [Salimans & Kingma, 2016]

○ Easier (than batch norm) to distribute over multiple GPUs

Lung Nodules

6

GTC EUROPE 2017

MR Lumbar Spine

● Foramen is the passage where a nerve exits the spine

● Foraminal stenosis is a common cause of leg pain

○ Time-consuming to find on scoliotic Spines

● Task: locate and classify all of them

Lumbar Foramina

7

GTC EUROPE 2017

Localization

● Foramina are big● Foramina are located at a certain

location● There are 10 lumbar foramina per spine

● Output a segmentation (probability map)● Requires less fine grained details

Lumbar Foramina

8

GTC EUROPE 2017

Localization architecture

● 3D U-Net architecture○ Easier to modify and/or extend○ Less need for efficient inference○ Input: 481x481x3

● Binary segmentation (probability map)

● Loss: (Fuzzy) Dice score○ Trains faster and requires less data for our

network than the normalized cross entropy loss

Lumbar Foramina

9

GTC EUROPE 2017

Lumbar Foramina

10

GTC EUROPE 2017

Lumbar Foramina

11

GTC EUROPE 2017

Combating over-generalization

● Reduce the resolution for more context● A segmentation per foramen level

● Bias towards the lower and bigger foramina

Lumbar Foramina

12

Recall L1 L2 L3 L4 L5

Base 0 0 0 0.97 0.96

GTC EUROPE 2017

Combating over-generalization

● Reduce the resolution for more context● A segmentation per foramen level

● Bias towards the lower and bigger foramina

● Normalize the Dice score

Lumbar Foramina

13

Recall L1 L2 L3 L4 L5

Base 0 0 0 0.97 0.96

Normalized 0.87 0.96 0.97 0.98 0.95

GTC EUROPE 2017

Lumbar Foramina

14

GTC EUROPE 2017

Lumbar Foramina

15

GTC EUROPE 2017

A Comparison

● Lung nodules:○ Challenge:

■ Detect small nodules in a vast volume■ Requires fine grained spatial details

○ Solution:■ Fully convolutional for efficient inference■ Dilated convolutions to keep network size in check■ Trains on 10,000s of samples

● Lumbar foramina ○ Challenge:

■ Detect big foramina in a (relatively) small volume■ Distinguish lumbar foramina from thoracic ones

○ Solution:■ Use 3D U-Net with Dice score■ Normalize the Dice score and reduce the resolution for increased performance■ Trains on 100s of samples

16

GTC EUROPE 2017

In practice

● FDA and CE clearance necessary for diagnostic impact○ Certify the training pipeline, inference pipeline, annotation tooling, deployment…○ Concept of independent test set matches very well○ Discussions on continuous learning (FDA, ACR)

● Aidence lung nodule detection submitted for CE○ Feedback received; clearance expected soon

17

GTC EUROPE 2017

“Your software has, in this short time, detected a patient with a nodule that has clearly grown in 3 years, is probably malignant and missed by 4 consecutive radiologists.”

- W.M, radiologist

18

markjan@aidence.com