Generation and Use of Quantitative Pathology Phenotype
-
Upload
joel-saltz -
Category
Health & Medicine
-
view
122 -
download
0
Transcript of Generation and Use of Quantitative Pathology Phenotype
![Page 1: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/1.jpg)
Generation and Use of Quantitative Pathology Phenotype
Joel Saltz MD,PhDChair Biomedical Informatics, Stony Brook
Associate Director Stony Brook Cancer Center
NCI Early Detection Research Network - Atlanta April 1, 2015
![Page 2: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/2.jpg)
Computational Pathology: High Dimensional Fused-Informatics
• Anatomic/functional characterization at fine and gross level
• Integrate of anatomic/functional characterization, multiple types of “omic” information, outcome
• Predict treatment outcome, select, monitor treatments
• High throughput tissue classification• Computer assisted exploration of new
classification schemes• Integrated analysis and presentation of
observations, features analytical results – human and machine generated
Pathology
Patient Outcome
Radiology
“Omic”Data
![Page 3: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/3.jpg)
Johns Hopkins School of Medicine
Virtual Microscope
1997 Proceedings AMIA Annual Meeting
![Page 4: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/4.jpg)
Information Technology for Cancer Research : NCI U24
Stony Brook: Joel Saltz, Tahsin Kurc, Yi Gao, Allen Tannenbaum, Fusheng Wang, Liangjia Zhu, Ivan Kolesov, Romeil Sandhu, Erich BremerEmory: Adam Marcus, Lee Cooper, Dan Brat, Fadlo Khuri Lee Cooper, Ashish Sharma, Rick Cummings, Roberd BostickOak Ridge National Lab: Scott Klasky, Dave PugmireYale: Michael Krauthammer
Tools to Analyze Morphology and Spatially Mapped Molecular Data
![Page 5: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/5.jpg)
Pathology Analytical Imaging
• Provide rich information about morphological and functional characteristics• Image analysis, feature extraction on multiple scales• Spatially mapped “omics”• Multiple microscopy modalities
Glass Slides Scanning Whole Slide Images Image Analysis
![Page 6: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/6.jpg)
Tumor Heterogeneity
Marusyk 2012
![Page 7: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/7.jpg)
Report: Imaging-Genomics Workshop June 2013
![Page 8: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/8.jpg)
Correlating Imaging Phenotypes with Genomic Signatures: Scientific Opportunities
Clinical Approach and Use• Development of imaging+analysis methods to characterize heterogeneity
• within a tumor at one time point• evolution over time• among different tumor types
• Development of imaging metrics that:• can predict and detect emergence of resistance?• correlates with genomic heterogeneity?• correlates with habitat heterogeneity?• can identify more homogeneous sub-types
Imaging Genomics Workshop NCI June 2013
![Page 9: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/9.jpg)
Direct Study of Relationship Between Image Features vs Clinical Outcome, Response to Treatment, Molecular Information
Lee Cooper,Carlos Moreno
![Page 10: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/10.jpg)
Integrative Morphology/”omics”
Quantitative Feature Analysis in Pathology: Emory In Silico Center for Brain Tumor Research (PI = Dan Brat, PD= Joel Saltz)
NLM/NCI: Integrative Analysis/Digital Pathology R01LM011119, R01LM009239 (Dual PIs Joel Saltz, David Foran)
Marcus Foundation Grant – Ari Kaufman, Joel Saltz
![Page 11: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/11.jpg)
Associations
![Page 12: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/12.jpg)
Gene Expression Correlates of GBM with High Oligo-Astro Ratio
Oligo Related Genes
Myelin Basic ProteinProteolipoproteinHoxD1
Nuclear features mostAssociated with OligoSignature Genes:
Circularity (high)Eccentricity (low)
![Page 13: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/13.jpg)
Pathology Computer Assisted Classification
Gurcan, Shamada, Kong, Saltz
Hiro Shimada, Metin Gurcan, Jun Kong, Lee Cooper Joel Saltz
![Page 14: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/14.jpg)
Neuroblastoma Classification
FH: favorable histology UH: unfavorable histologyCANCER 2003; 98:2274-81
<5 yr
SchwannianDevelopment
≥50%Grossly visible Nodule(s)
absent
present
Microscopic Neuroblastic
foci
absent
present
Ganglioneuroma(Schwannian stroma-dominant)
Maturing subtypeMature subtype
Ganglioneuroblastoma, Intermixed(Schwannian stroma-rich)
FH
FH
Ganglioneuroblastoma, Nodular(composite, Schwannian stroma-rich/stroma-dominant and stroma-poor) UH/FH*
Variant forms*
None to <50%
Neuroblastoma(Schwannian stroma-poor)
Poorly differentiatedsubtype
Undifferentiatedsubtype
Differentiatingsubtype
Any age UH
≥200/5,000 cellsMitotic & karyorrhectic cells
100-200/5,000 cells
<100/5,000 cells
Any age
≥1.5 yr
<1.5 yr
UH
UH
FH
≥200/5,000 cells
100-200/5,000 cells
<100/5,000 cells
Any age UH
≥1.5 yr
<1.5 yr
≥5 yr
UH
FH
UH
FH
![Page 15: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/15.jpg)
Computerized Classification System for Grading Neuroblastoma
• Background Identification
• Image Decomposition (Multi-resolution levels)
• Image Segmentation (EMLDA)
• Feature Construction (2nd order statistics, Tonal Features)
• Feature Extraction (LDA) + Classification (Bayesian)
• Multi-resolution Layer Controller (Confidence Region)
No
YesImage Tile Initialization
I = L Background? Label
Create Image I(L)
Segmentation
Feature Construction
Feature Extraction
Classification
Segmentation
Feature Construction
Feature Extraction
Classifier Training
Down-sampling
Training Tiles
Within ConfidenceRegion ?
I = I -1
I > 1?
Yes
Yes
No
No
TRAINING
TESTING
![Page 16: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/16.jpg)
![Page 17: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/17.jpg)
Large Scale Data Management
Data model capturing multi-faceted information including markups,annotations, algorithm provenance, specimen, etc.Support for complex relationships and spatial query: multi-level
granularities, relationships between markups and annotations, spatialand nested relationshipsHighly optimized spatial query and analysesImplemented in a variety of ways including optimized CPU/GPU,
Hadoop/HDFS and IBM DB2 (Wang, Saltz, Kurc)
NLM/NCI: Integrative Analysis/Digital Pathology R01LM011119, R01LM009239 (Dual PIs Joel Saltz, David Foran)
![Page 18: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/18.jpg)
Spatial Centric – Pathology Imaging “GIS”Point query: human marked point inside a nucleus
.
Window query: return markups contained in a rectangle
Spatial join query: algorithm validation/comparison
Containment query: nuclear featureaggregation in tumor regions
Fusheng Wang
![Page 19: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/19.jpg)
HPC Pipelines (Segmentation and Feature Extraction)
Tony Pan, George Teodoro,Tahsin Kurc and Scott Klasky
![Page 20: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/20.jpg)
Tools to Analyze Morphology and Spatially Mapped Molecular Data1U24CA180924-01
SPECIFIC AIMS
![Page 21: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/21.jpg)
Pathology Image Analysis
• Specific Aim - Develop, deploy, and evaluate robust and scalable methods and analysis pipelines for multi- scale, integrative image analysis.
• Aim 1a Develop methods to segment micro-anatomic objects and extract and classify features from whole slide tissue images.
• Aim 1b Develop analytic methods to carry out spatially mapped molecular tissue characterizations.
• Aim 1c Develop methods to create 3-D reconstructions of multi- scale micro-anatomic features and to characterize changes in morphology over time.
![Page 22: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/22.jpg)
Pipelines, Database, Data modeling, Visualization
• Specific Aim 2: Develop database infrastructure to manage and query image data, image analysis results.
• Specific Aim 3: Develop high performance software that targets clusters, cloud computing, and leadership scale systems.
• Specific Aim 4: Develop visualization middleware for 2D/3D image and feature data and for integrated image and “omic” data.
![Page 23: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/23.jpg)
caMicroscope - A Digital Pathology Integrative Query System – PI Ashish Sharma -CTIIP IIWG
• Interoperate with U24 MongoDB feature database to allow annotation and markup view and generation
• Extensions to support creation and display of annotations (free form pencil tool, polygon tool, rectangle & ellipse)
• Measurement tool & magnifying glass (magnify by 200% a region of interest)
![Page 24: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/24.jpg)
caMicroscope/MongoDB - Multiple Algorithm Comparison
![Page 25: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/25.jpg)
Why we need multiple algorithm comparison
![Page 26: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/26.jpg)
Hierarchical Pipeline (Slicer!)Liangjia ZhuIvan KolesovRomeil SandhuAllen Tannenbaum
![Page 27: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/27.jpg)
Low Power
• Fast GrowCut segmentation• Intensity insufficient: need user
guidance• Boundaries are most time
consuming for user
![Page 28: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/28.jpg)
Medium Power
• Adaptive thresholdingsegmentation
• Allow for global user input (influence parameter settings)
![Page 29: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/29.jpg)
Crypt/Nuclear Segmentation
• Variational active contour• Context is crucial
![Page 30: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/30.jpg)
Convolutional Neural Network Classification
Le Hou, Dimitris Samaras, Tahsin Kurc, Yi Gao, Liz Vanner, James Davis, Joel Saltz
![Page 31: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/31.jpg)
![Page 32: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/32.jpg)
Confocal/Super resolution nuclear morphometry (Slicer!)Ken Shroyer, Yi Gao, Tahsin Kurc, Joel Saltz • Pancreatic Fine Needle
Aspirate• Correlative studies
linking fine needle aspirate cell data, “omic” and Radiology imaging data
• Leverages Marcus foundation virtual biopsy effort
![Page 33: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/33.jpg)
Cells first prepared via Papanicolaou stain – identified as not suspicious
Preliminary Work
![Page 34: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/34.jpg)
Cells first prepared via Papanicolaou stain – identified as suspicious
![Page 35: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/35.jpg)
Cytology shape analysis
volume elongation surface area roundness sp-radius
sp-area ellipse diameter axis 1 ellipse diameter axis2 ellipse diameter axis 3 flatness
![Page 36: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/36.jpg)
3D Imaging Analytics• 3D Vessel Reconstruction with Serial Microscopy Images
• WSIs from serial sections have significant potential to enhance the study of both normal and disease processes
• 3D reconstruction of cellular level objects is a critical step
(Yanhui Liang, Jun Kong, Fusheng Wang, Darren Treanor – Wang NSF CAREER Award)
![Page 37: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/37.jpg)
3D Imaging Analytics: Vessel Association
1
2
2 1
(a)
(b)
(a) Panoramic view of 3D reconstructed vessels; (b) two close-up views ofsegment 1 and 2 indicated in the panoramic view.
![Page 38: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/38.jpg)
Driving Cancer Research, Community Support, Challenges and Engagement
• Specific Aim 5: Drive continuing development of the tools using a suite of cancer driving biomedical problem, and provide collaborative support and training to the cancer research community.
• Aim 5a: Evaluate, demonstrate and drive continuing development of the tools using a suite of cancer driving biomedical problems.
• Aim 5b: Provide support to the cancer research community through: • Engagement in NCI Quantitative Imaging Network• support of community digital Pathology image analysis “grand challenges” – initial
challenge involving analysis of TCGA whole slide image at MICCAI 2014,• partnerships with collaborative efforts described in letters of support and collaboration,
including the National Cancer Imaging Archive, the Mayo Quantitative Imaging Network site, the Colon Cancer Family Registry and the Polyp Prevention Study,
• partnerships with cancer microscopy/Pathology shared resources • development of on line resources and workshops to teach users how to employ U24
tools.
![Page 39: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/39.jpg)
CORRELATIVE FEATURE ANALYSIS
![Page 40: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/40.jpg)
MICCAI 2014 BRAIN TUMORClassification and Segmentation Challenges
TCGA
TCIA
IMAGING CHALLENGE
DIGITAL PATHOLOGY CHALLENGE
Phase 1: TrainingJune 20 - July 31
Phase 2: Leader BoardAug 1 - Aug 29
Phase 3: TestSept 8 - Sept 12
For more information about these challenges and a related workshop on September 14, 2014 at MICCAI in Boston, see: cancerimagingarchive.net
MICCAI: Medical Image Computing and Computer Aided Interventions - MICCAI2014.orgTCGA: The Cancer Genome Atlas - cancergenome.nih.govTCIA: The Cancer Image Archive - cancerimagingarchive.net
![Page 41: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/41.jpg)
Digital Pathology/Brain Tumor Image Segmentation (BRATS)
• Used data currently available through data archive resources of the National Institutes of Health (NIH), namely, the Cancer Genome Atlas (TCGA) and the Cancer Image Archive (TCIA)
• Digital Pathology challenge used digital slides related to patients whose genomics data are available from TCGA. Similarly, BRATS 2014 Challenge used clinical MRI image data, also from the TCGA study subjects.
• Sub-Challenge 1: Classification - Automated classification of LGG and GBM from a collection of 30+ high-resolution digital pathology slides.
• Sub-Challenge 2: Segmentation – Automated segmentation of necrotic and normal brain regions on regions of digital pathology slides from a collection of 20+ GBM cases.
![Page 42: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/42.jpg)
Organizers and Major Contributors
• Daniel J. Brat, Emory University• Larry Clarke, National Cancer Institute• James Davis, Stony Brook Cancer Center• Keyvan Farahani, National Cancer Institute• John Freymann, Leidos Biomedical Res, Inc.• Carl Jaffe, Boston University• Justin Kirby, Leidos Biomedical Res., Inc.• Tahsin Kurc, Stony Brook Cancer Center• Miguel Ossandon, National Cancer Institute• Joel Saltz, Stony Brook Cancer Center• Roberta Seidman, Stony Brook Cancer Center
![Page 43: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/43.jpg)
Example of a “good match”
![Page 44: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/44.jpg)
Example of a problematic match
![Page 45: Generation and Use of Quantitative Pathology Phenotype](https://reader030.fdocuments.in/reader030/viewer/2022032513/55d2d0dcbb61eb996f8b47b8/html5/thumbnails/45.jpg)
Thanks!