Pathomics Based Biomarkers and Precision Medicine
-
Upload
joel-saltz -
Category
Health & Medicine
-
view
79 -
download
2
Transcript of Pathomics Based Biomarkers and Precision Medicine
![Page 1: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/1.jpg)
Pathomics Based Biomarkers and Precision Medicine
Joel SaltzDepartment of Biomedical Informatics
Stony Brook University
Imaging 2020Sept 19, 2016
![Page 2: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/2.jpg)
Multi-Scale Precision Medicine
![Page 3: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/3.jpg)
Multi-scale Integrative Analysis in Biomedical Informatics
• Predict treatment outcome, select, monitor treatments
• Reduce inter-observer variability in diagnosis
• Computer assisted exploration of new classification schemes
• Multi-scale cancer simulations
![Page 4: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/4.jpg)
Pathomics, Radiomics
Identify and segment trillions of objects – nuclei, glands, ducts, nodules, tumor niches … from Pathology, Radiology imaging datasetsExtract features from objects and spatio-temporal regionsSupport queries against ensembles of features extracted from multiple datasetsStatistical analyses and machine learning to link Radiology/Pathology features to “omics” and outcome biological phenomenaPrinciple based analyses to bridge spatio-temporal scales – linked Pathology, Radiology studies
![Page 5: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/5.jpg)
Radiomics
Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach
Hugo J. W. L. Aerts et. Al.Nature Communications 5, Article number: 4006 doi:10.1038/ncomms5006
Features
Patients
![Page 6: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/6.jpg)
Integrative Morphology/”omics”
Quantitative Feature Analysis in Pathology: Emory In Silico Center for Brain Tumor Research (PI = Dan Brat, PD= Joel Saltz)
NLM/NCI: Integrative Analysis/Digital Pathology R01LM011119, R01LM009239 (Dual PIs Joel Saltz, David Foran)
J Am Med Inform Assoc. 2012 Integrated morphologic analysis for the identification and characterization of disease subtypes.
Pathomics
Lee Cooper, Jun Kong
![Page 7: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/7.jpg)
Consensus clustering of morphological signatures
Study includes 200 million nuclei taken from 480 slides corresponding to 167 distinct patients
Each possibility evaluated using 2000 iterations of K-means to quantify co-clustering
Nuclear Features Used to Classify GBMs
3 2 1
20 40 60 80 100 120 140 160
20
40
60
80
100
120
140
1602 3 4 5 6 725
30
35
40
45
50
# Clusters
Silh
ouet
te A
rea
0 0.5 1
1
2
3
Silhouette Value
Clu
ster
![Page 8: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/8.jpg)
Clustering identifies three morphological groups• Analyzed 200 million nuclei from 162 TCGA GBMs (462 slides)
• Named for functions of associated genes: Cell Cycle (CC), Chromatin Modification (CM), Protein Biosynthesis (PB)
• Prognostically-significant (logrank p=4.5e-4)Fe
atur
e In
dice
s
CC CM PB
10
20
30
40
500 500 1000 1500 2000 2500 3000
0
0.2
0.4
0.6
0.8
1
Days
Sur
viva
l
CCCMPB
![Page 9: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/9.jpg)
Associations
![Page 10: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/10.jpg)
Oligodendroglioma Astrocytoma
Nuclear Qualities
TCGA GBM Whole Slide Images – Pathomics Collaborations with Dan Brat, Lee Cooper, Emory
Application: Clinical and Molecular Significance of Oligodendroglioma Component in GBM
![Page 11: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/11.jpg)
Gene Expression Correlates of High Oligo-Astro Ratio on Machine-based Classification
Oligo Related Genes
Myelin Basic ProteinProteolipoproteinHoxD1
Nuclear features mostAssociated with Oligo Signature Genes:
Circularity (high)Eccentricity (low)
![Page 12: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/12.jpg)
Tools to Analyze Morphology and Spatially Mapped Molecular Data - U24 CA180924 • Specific Aim 1 Analysis pipelines for multi- scale,
integrative image analysis.• Specific Aim 2: Database infrastructure to manage
and query Pathomics features.• Specific Aim 3: HPC software that targets clusters,
cloud computing, and leadership scale systems.• Specific Aim 4: Develop visualization middleware
to relate Pathomics feature and image data and to integrate Pathomics image and “omic” data.
![Page 13: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/13.jpg)
Robust Nuclear Segmentation
• Robust ensemble algorithm to segment nuclei across tissue types• Optimized algorithm tuning methods• Parameter exploration to optimize quality• Systematic Quality Control pipeline encompassing tissue image
quality, human generated ground truth, convolutional neural network critique
• Yi Gao, Allen Tannenbaum, Dimitris Samaras, Le Hou, Tahsin Kurc
![Page 14: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/14.jpg)
Cell Morphometry Features
![Page 15: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/15.jpg)
![Page 16: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/16.jpg)
Feature Explorer Suite
• Explore Relationship Between Imaging Features, Outcome, ”omics”
• Explore relationships between features and explore how features relate to images
![Page 17: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/17.jpg)
Feature Explorer - Integrated Pathomics Features, Outcomes and “omics” – TCGA NSCLC Adeno Carcinoma Patients
![Page 18: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/18.jpg)
Feature Explorer - Integrated Pathomics Features, Outcomes and “omics” – TCGA NSCLC Adeno Carcinoma Patients
![Page 19: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/19.jpg)
Collaboration with MGH – Feature Explorer – Radiology Brain MR/Pathology Features
![Page 20: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/20.jpg)
Collaboration with SBU Radiology – TCGA NSCLC Adeno Carcinoma
Integrative Radiology, Pathology, “omics”, outcome
Mary Saltz, Mark Schweitzer SBU Radiology
![Page 21: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/21.jpg)
PathomicsRelationship Between Image and Features
![Page 22: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/22.jpg)
![Page 23: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/23.jpg)
Select Feature Pair – dots correspond to nuclei
![Page 24: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/24.jpg)
Subregion selected – form of gating analogous to flow cytometry
![Page 25: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/25.jpg)
Sample Nuclei from Gated Region
![Page 26: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/26.jpg)
Gated Nuclei in Context
![Page 27: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/27.jpg)
Algorithm Comparison, Validation, Uncertainty Quantification• High quality image analysis algorithms are essential to
support biomedical research and diagnosis• Validate algorithms with human annotations• Compare and consolidate different algorithm results
e.g.: what are the distances and overlap ratios between markup boundaries from two algorithms?
Green: algorithm 1Red: algorithm 2
Cross matching of two spatial data sets
MICCAI 2014 Brain Tumor
Introduction
![Page 28: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/28.jpg)
caMicroscope/MongoDB - Multiple Algorithm Comparison; Generate and Curate Pathomics Feature set
![Page 29: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/29.jpg)
Compare Algorithm Results
![Page 30: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/30.jpg)
Heatmap – Depicts Agreement Between Algorithms
![Page 31: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/31.jpg)
3D Slicer Pathology – Generate High Quality Ground Truth
![Page 32: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/32.jpg)
Apply Segmentation Algorithm
![Page 33: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/33.jpg)
Adjust algorithm parameters, manual fine tuning
![Page 34: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/34.jpg)
Classification• Automated or semi-automated identification of
tissue or cell type• Variety of machine learning and deep learning
methods• Classification of Neuroblastoma• Classification of Gliomas• Quantification of lymphocyte infiltration
![Page 35: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/35.jpg)
Classification and Characterization of Heterogeneity
Gurcan, Shamada, Kong, Saltz
Hiro Shimada, Metin Gurcan, Jun Kong, Lee Cooper Joel Saltz
BISTI/NIBIB Center for Grid Enabled Image Analysis - P20 EB000591, PI Saltz
![Page 36: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/36.jpg)
Neuroblastoma Classification
FH: favorable histology UH: unfavorable histologyCANCER 2003; 98:2274-81
<5 yr
SchwannianDevelopment
≥50%Grossly visible Nodule(s)
absent
present
Microscopic Neuroblastic
foci
absent
present
Ganglioneuroma(Schwannian stroma-dominant)
Maturing subtypeMature subtype
Ganglioneuroblastoma, Intermixed(Schwannian stroma-rich)
FH
FH
Ganglioneuroblastoma, Nodular(composite, Schwannian stroma-rich/stroma-dominant and stroma-poor) UH/FH*
Variant forms*
None to <50%
Neuroblastoma(Schwannian stroma-poor)
Poorly differentiatedsubtype
Undifferentiatedsubtype
Differentiatingsubtype
Any age UH
≥200/5,000 cellsMitotic & karyorrhectic cells
100-200/5,000 cells
<100/5,000 cells
Any age
≥1.5 yr
<1.5 yr
UH
UH
FH
≥200/5,000 cells
100-200/5,000 cells
<100/5,000 cells
Any age UH
≥1.5 yr
<1.5 yr
≥5 yr
UH
FH
UH
FH
![Page 37: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/37.jpg)
Multi-Scale Machine Learning Based Shimada Classification System
• Background Identification
• Image Decomposition (Multi-resolution levels)
• Image Segmentation (EMLDA)
• Feature Construction (2nd order statistics, Tonal Features)
• Feature Extraction (LDA) + Classification (Bayesian)
• Multi-resolution Layer Controller (Confidence Region)
No
YesImage Tile Initialization
I = L Background? Label
Create Image I(L)
Segmentation
Feature Construction
Feature Extraction
Classification
Segmentation
Feature Construction
Feature Extraction
Classifier Training
Down-sampling
Training Tiles
Within ConfidenceRegion ?
I = I -1
I > 1?
Yes
Yes
No
No
TRAINING
TESTING
![Page 38: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/38.jpg)
![Page 39: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/39.jpg)
Glioma classification using convolutional neural networks
Le Hou, Dimitris Samaras, Tahsin Kurc, Yi Gao, Liz Vanner, James Davis, Joel Saltz
![Page 40: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/40.jpg)
Tumor Infiltrating Lymphocyte quantification• Convolutional neural
network to classify lymphocyte infiltration in tissue patches
• Convolutional neural network and random forest to classify individual segmented nuclei
• Extensive collection of ground truth
• In progress, joint work with Emory and TCGA PanCanAtlas Immune group
![Page 41: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/41.jpg)
Dissemination
• Containers• Cloud• TCIA• HPC via NSF and DOE• TCGA – PanCanAtlas – Lymphocyte characterization• Integrated Features/NLP joint with TIES
![Page 42: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/42.jpg)
ITCR TeamStony Brook UniversityJoel SaltzTahsin KurcYi GaoAllen TannenbaumErich BremerJonas AlmeidaAlina JasniewskiFusheng WangTammy DiPrimaAndrew WhiteLe HouFurqan BaigMary Saltz
Emory UniversityAshish SharmaAdam Marcus
Oak Ridge National LaboratoryScott KlaskyDave PugmireJeremy Logan
Yale UniversityMichael Krauthammer
Harvard University Rick Cummings
![Page 43: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/43.jpg)
Funding – Thanks!• This work was supported in part by U24CA180924-
01, NCIP/Leidos 14X138 and HHSN261200800001E from the NCI; R01LM011119-01 and R01LM009239 from the NLM
• This research used resources provided by the National Science Foundation XSEDE Science Gateways program under grant TG-ASC130023 and the Keeneland Computing Facility at the Georgia Institute of Technology, which is supported by the NSF under Contract OCI-0910735.
![Page 44: Pathomics Based Biomarkers and Precision Medicine](https://reader035.fdocuments.in/reader035/viewer/2022070511/58a3ec341a28ab272e8b58cf/html5/thumbnails/44.jpg)
Thanks!