AUTOMATIC SEGMENTATION OF PULMONARY LOBES USING A PROGRESSIVE DENSE V-NETWORK

Abdullah-Al-Zubaer Imran1,2, Ali Hatamizadeh1,2, Shilpa P. Ananth2, Xiaowei Ding1,2, Demetri Terzopoulos1,2, Nima Tajbakhsh2

1University of California, Los Angeles

2VoxelCloud, Inc.

Clinical Background and Motivation

• Automated radiology report generation• e.g., emphysema found in RUL and LUL

• Nodule localization• e.g., 3 mm nodule in the left upper lobe

• Treatment planning• e.g., lung volume reduction surgery (LVRS)

• Automatic and efficient pulmonary lobe segmentation is important

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network2

Successful LVRS: Postoperative CT showing aerated lung of the lower lobes[Yanagisawa et al. 2013]

Nodule in the left upper lobe[qualityhealthcareplease.wordpress.com]

Emphysema in the upper lobes[Sukumar et al. 2008]

Lung Lobes and Fissures

• Folding of visceral pleura creates the major (oblique) and minor (horizontal) fissures

• The major and minor fissures define the lobar boundaries

• Each functionally independent lobe has separate bronchial and vascular systems

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network3

Coronal slice of lung CT showing the lobes and fissures

Challenges

• Fissures are often incomplete• May not extend to the lobar boundaries

• Visible fissures might not be enough for distinguishing the lobes

• Fissures can vary in thickness, location, and shape even when they are visible

• Other fissures can be misinterpreted as major or minor fissures• E.g., azygos fissures and accessory

fissures

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network4

Accessory fissure Azygos fissure

Incomplete fissure

Related Work

• Dependence on priors• Good accuracy• Pulmonary vessel and airway segmentations in prior [Bragman et al. 2017]

• Semi-automatic segmentations• Less dependency on the quality of priors• Fissure initialization [Doel et al. 2012]

• Manually-defined atlas • Laborious to create• Higher execution time [Ross et al. 2010]• Poor performance in highly variable pathological lungs

• 2D FCN followed by random walker• Not end-to-end; reliant on the subsequent heuristic method for optimal results• Slow run-time (4-8 min per case) [George et al. 2017]

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network5

Deficiencies of Existing Techniques

• The existing techniques are • Prohibitively slow,

• Undesirably reliant on prior (airway/vessel) segmentation, and/or

• Require user interactions for optimal results

• Can we have a model which is• Fast

• Robust against any CT scan cases

• End-to-end

• Non-reliant on any prior air-way/vessel segmentation

• Independent on anatomical information, or atlases?

• Our work fills all the above gaps!

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network6

Key Contributions

• The first ever end-to-end 3D CNN-based lobe segmentation model

• The fastest lobe segmentation model• Each inference takes only 2 seconds

• Fully automatic• Does not rely on any prior segmentation

• Robust against a variety of CT scan configurations and pathologies

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network7

Proposed Method: PDV-Net

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network8

Baseline Methods

• U-Net architecture (2D)• Most recently published article on lobe segmentation

• Dense V-Net (3D)• A strong baseline for comparison

• Basically used for abdominal segmentation

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network9

Implementation Details

• PDV-Net and baseline dense V-Net:• Training volumes first normalized, followed by rescaling to 512x512x64

• CPU: Intel(R) Xeon(R) CPU E5-2697 v4@2.30GHz machine

• GPU: 1 Nvidia Titan XP

• Adam optimizer with a learning rate of 0.01 and a weight decay of 1e-7

• U-Net:• Axial slices from all the training volumes, each sized 512x512

• Only slices wherein at least one lung lobe is present

• Adam optimizer with a learning rate of 5-5 and batches of 10 images

• Activation: PReLU

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network10

Dataset and Ground Truth

• Training• 270 cases from the LIDC dataset

• Testing• 84 LIDC cases

• 154 LTRC cases

• 55 LOLA11 challenge cases

• Ground truth segmentation• Chest Imaging Platform feature on 3D Slicer

• Fiducial points for the major and minor fissures

• Segmentation masks were later quality checked

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network11

Interactive lobe segmentation[chestimagingplatform.org]

Results

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network12

CT Slice GT U-Net DV-Net PDV-Net

Left

Lung

Right

Lung

3D

Results (cont’d)

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network13

Results (cont’d)

LOLA11 (55)

Lobe Mean ± SD Q1 Median Q3

RUL 0.9518 ± 0.1750 0.9371 0.9688 0.9881

RML 0.8621 ± 0.4149 0.8107 0.9284 0.9663

RLL 0.9581 ± 0.1993 0.9621 0.9829 0.9881

LUL 0.9551 ± 0.2160 0.9644 0.9834 0.9924

LLL 0.9342 ± 0.3733 0.9546 0.9805 0.9902

Overall 0.9345

Bragman et al. 0.9384

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network14

Robustness Analysis

• A good agreement between our segmentation model and ground truth

• 84 LIDC test cases were grouped based on• Reconstruction kernel

• soft, lung, and bone

• Size of reconstruction interval• Z-spacing≤1, 1<Z-spacing<2, and Z-spacing≥2

• CT scan vendors• GE, Philips, Siemens, and Toshiba

• No significant differences were observed as confirmed by the one-way ANOVA

• Lobe segmentation accuracy is not correlated with emphysema index (Pearson correlation)

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network15

Limitations

• Segmentation accuracy for the right middle lobes is low compared to the other lobes

• Performance for the challenging cases could still be an issue• e.g., LOLA case (in the figures)

• With some minor preprocessing and post-processing, by smoothing the results, better accuracy may be achieved

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network16

Conclusions

• Automatic, fast, and reliable segmentation of pulmonary lobes

• The proposed model outperforms, or at worst performs comparably to, the state-of-the-art

• Robustness of the model against varying configurations of CT reconstruction, choice of CT vendor, and presence of lung pathologies

9/20/2018Automatic Segmentation of Pulmonary Lobes

Using a Progressive Dense V-Network17

AUTOMATIC SEGMENTATION OF PULMONARY LOBES USING A PROGRESSIVE DENSE V-NETWORK

Abdullah-Al-Zubaer Imran1,2, Ali Hatamizadeh1,2, Shilpa P. Ananth2, Xiaowei Ding1,2, Demetri Terzopoulos1,2, Nima Tajbakhsh2

1University of California, Los Angeles

2VoxelCloud, Inc.

QUESTIONS?