Using Deep Learning for Earlier Detection of Acute ... · Algorithm 2 Performance on Stroke: 62%...
Transcript of Using Deep Learning for Earlier Detection of Acute ... · Algorithm 2 Performance on Stroke: 62%...
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
1
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 2017
Using Deep Learning for Earlier Detection of Acute
Infarction of the Brain
Barbaros S. Erdal, Ph.D.
Department of Radiology
The Ohio State University Wexner Medical Center
May, 2017
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
2
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 2017
2What is New in the Field: Radiology
Positive
(Gold standard)
Negative
(Gold Standard)
Total
Positive
(AI algorithm)
38 5 43
Negative
(AI Algorithm)
2 35 37
Total 40 40 80
Neurologic Disease Medical Imaging Informatics
(e.g., Artificial Intelligence)
Cardiovascular DiseaseFast MRI
(e.g., MRE, 4D Flow)
Cancer Low-Dose MolecuIar Imaging
(e.g., Digital PET)
Standard 10 x Reduction
NCI R01CA195513
RSNA Medical Student
RSNA Molecular Imaging
Ohio Third Frontier
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
3
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20173
Explore “Deep Learning” for Pattern Recognition inImages, Digital Pathology, and Genomic Data:
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
4
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20174
Radiology Workflow
EMR System
AgeGenderReason
CPOE for Radiology Exam
Physician requests imagingstudy
Imaging Exam Requesting
Facility
REMIX
HIS/RIS
Exam Scheduled and Performed
PACSReconstructed images are searchable, and ready foradvanced Image analysis
All study relevant data and Images are linkable andmineable
Clinical Reasearchers
REMIX receives and de-identifiesimage data and related metadata
Data Warehouse
Patients have already beenConsented for TCCP
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
5
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20175
Physician requests imagingstudy
Imaging Exam Requesting
Facility
REMIX Desktop
PACS
Reconstructed images are searchable, and ready foradvanced Image analysis
All study relevant data, Images are linkable andmineable
Clinical Reasearchers / Researchers
REMIX receives and de-identifiesimage data and related metadata
Scanner
Enterprise Data Warehouse
Patients have already beenConsented based on study involved
Diagnostic Workstation
EMR\HIS\RISEnterprise Viewer
VNA
REMIX Recon REMIX BIREMIX AI
REMIX Server
Clinical\Operational User
Clinical and Operationaluses are uninterrupted
• REMIX (Radiology and Enterprise Medical Imaging Extensions)
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
6
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20176
• REMIX: De-Identification (Operated by OSUWMC Imaging Informatics)
Patient Search for Research Dataset Preparation
Clinical Trial Support
Honest Broker Compliant Batch Image Processing for Large datasets
Data verification supported by OSUWMC Imaging Informatics
CD Burning and Image sending to custom folders and\or destinations
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
7
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20177
REMIX Data-Mining Capabilities
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
8
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20178
REMIX Data-Mining Capabilities
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
9
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 20179
REMIX: Quantitative Capabilities
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
10
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201710
Developed Texture-Analysis Capabilities
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
11
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201711
Texture Analysis Capabilities
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
12
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 2017
12Quality Metrics: Radiology - Clinical Service Efficiency
Source: IHIS
Neuro MRI: Routine vs Stat
Stat
Role for Artificial Intelligence in Imaging
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
13
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201713
Physician requests imagingstudy
Imaging Exam Requesting
Facility
REMIX Desktop
PACS
Reconstructed images are searchable, and ready foradvanced Image analysis
All study relevant data, Images are linkable andmineable
Clinical Reasearchers / Researchers
REMIX receives and de-identifiesimage data and related metadata
Scanner
Enterprise Data Warehouse
Patients have already beenConsented based on study involved
Diagnostic Workstation
EMR\HIS\RISEnterprise Viewer
VNA
REMIX Recon REMIX BIREMIX AI
REMIX Server
Clinical\Operational User
Clinical and Operationaluses are uninterrupted
• REMIX
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
14
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201714
Positive
(Gold standard)
Negative
(Gold Standard)
Total
Positive
(AI algorithm)
38 5 43
Negative
(AI Algorithm)
2 35 37
Total 40 40 80
Neurologic Disease Medical Imaging Informatics
(e.g., Artificial Intelligence)
REMIX -AI
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
15
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201715
REMIX AI specs
Processor 1x intel Core i7-5930 K Processor (15M Cache,
3.50 GHz)
Memory 64GB DDR4
GPUs 4 x NVIDIA GeForce GTX Titan X GPUs (7
Teraflops of single precision, 336.5 GB/s of
memory bandwidth, 12 GB memory per GPU)
Operating System (OS) Ubuntu 14.04
Storage 2x 256 GB SSD disk for OS and software libraries
and 3x3TB standard disk on RAID 5 for data
storage
Connecting to REMIX AI:1) REMIX AI web interface, allowing users to upload their data into the
system2) REMIX Desktop, permitting users to directly save their image data
into shared disk drives of REMIX AI3) Python-based client libraries, so that users can make Restful API calls
to REMIX PACS
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
16
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201716
REMIX BI (Dataset Queries)
1,015
611
104 101 89
980
582
75 68 60
951
564
58 52 450
200
400
600
800
1,000
1,200
2008 2010 2014 2015 2016
Order to Finalize in Minutes Begin to Finalize in Minutes
Complete to Finalize in Minutes Expon. (Complete to Finalize in Minutes)
Query 1: Find all “non-contrast head CT Exams” where in the clinical system patients has been associated with “critical findings” (e.g., hemorrhage, mass effect and hydrocephalus) for a given month (under 12 seconds)
Query 2: Find all “non-contrast Head CT Exams” where in the clinical system patients have been associated with “Stroke” (under 8 seconds)
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
17
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201717
REMIX - Desktop
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
18
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201718
REMIX - Recon
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
19
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201719
Pitfalls
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
20
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201720
REMIX Desktop: Pre-Processing DICOM to JPEG
Installed on a desktop system: Intel Xeon E3-1270 v5 @ 3.60 GHz, 4 Cores CPU, 32 GB installed system memory, and an NVIDIA Quadro K1200 GPU with 4 GB graphics memory.
DICOM to color jpeg images using color lookup tables for each respective window and level.
Query 1, a “Brain Window” setting was utilized (W90/C40).
Query 2 a “Stroke Window’ setting was utilized (W30/C30).
The results were written into shared folders where they could be accessed by the REMIX AI module.
243 minutes.
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
21
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201721
Data Flow for AI
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
22
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201722
Results
True Positive example for Algorithm 1
Algorithm 1 Performance on Hemorrhage, Mass effect, or hydrocephalus: 90% Sensitivity (CI95%, 78-97%). 85% Specificity (CI95%, 76-92%). AUC = 0.91
Algorithm 2 Performance on Stroke: 62% Sensitivity (CI95%, 38-82%). 96% Specificity (CI95%, 82-100%), with AUC = 0.81
True Positive example for Algorithm 2
False Negative example for Algorithm 1
False Positive example for Algorithm 2
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
23
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201723
REMIX AI Performance
Images to 256x256 matrix and processed with GoogLeNet convolutional network running on Caffe. 60 training epochs used.
Model creation with the first dataset (from Query 1, with 2,583 images) was 6 minutes and 19 seconds
Model creation for the second dataset (from Query 2, with 646 images), total processing took 97 seconds
Once image-classification models were created, batch image classifications performed at approximately 25 images per second.
Confidential │ Trade Secret │ Proprietary │ Do Not Copy
24
Confidential │ Trade Secret │ Proprietary │ Do Not Copy Strategy and Planning │The Ohio State University Wexner Medical Center © 201724
Questions?
Barbaros S. Erdal, Ph.D.
Department of RadiologyThe Ohio State University Wexner Medical Center
Contact: [email protected]