24th February 20094th Health IT Summer Showcase1 4 th Summer Health IT Showcase -2009 Health...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of 24th February 20094th Health IT Summer Showcase1 4 th Summer Health IT Showcase -2009 Health...
24th February 2009 4th Health IT Summer Showcase 1
4th Summer Health IT Showcase -2009
Health Information Technologies Research Laboratory
School of IT
University of Sydney
24th February 2009 4th Health IT Summer Showcase 2
HITRL Objectives• Research in Natural Language processing
for medical content
• Research into Clinical Information Systems
• Research into the use of terminologies -SNOMED CT, Apache IV, NIC, NOC, etc.
• We are learning how to build such functionality
24th February 2009 4th Health IT Summer Showcase 3
Current Enhancement Technologies
• Clinical Data Analytics Language (CliniDAL)
• Generative Clinical Information Management Systems (GCIMS)
• Bolt-on to existing systems
24th February 2009 4th Health IT Summer Showcase 4
1st Session
• Pathology Research and Clinical Data Analytics Language - Cleverer interfaces for research on clinical databases
• Visual Annotator - getting at the correct language - clinical notes, synoptic reports - automating annotation
• Generating Interoperable Clinical Information Systems - Multidisciplinary & Nursing CIS, Trauma CIS
• Intelligent Notes System - Automatic text correction with information retrieval on the ward rounds
Health Information Technologies Research Laboratory
24th February 2009 4th Health IT Summer Showcase 5
2nd Session• Identifying SNOMED CT codes in clinical
notes
• Identifying medical concepts in published papers
• Recognising co-morbidities in clinical notes of Obesity patients
• Unpacking clinical notes - recognising words and non-words
24th February 2009 4th Health IT Summer Showcase 6
Other Projects in 2008• Semester 1
– CLINIDAL installed on SWAPS AP data warehouse– Pathology Classification on SWAPS AP database– Handovers generating system for the ICU
• Semester 2– Graphical viewer for SNOMED CT on Term Server– Workflow on General Medical Wards - BMDH– Software testing for CliniDAL– CLINIDAL installed in CareVue data warehouse– Intelligent Notes for ICU
24th February 2009 4th Health IT Summer Showcase 7
Structured Reporting for Pathology Results
• Melanoma - supported by QUPP in collaboration with Dr Richard Scolyer and Dr Raj Murali at the RPAH
• Breast Cancer - supported by the BCI Westmead with Dr John Boyages and Dr Nehmat Houssami
24th February 2009 4th Health IT Summer Showcase 8
Melanoma-Executive Summary • Project objectives achieved• 477 histopathology reports annotated for 22
concepts of data by four annotators• Linguists missed 6.0% on average of
pathologists labels• 19 fields appear to be reliably computable• Gold-standard set of reports assembled• By-products: advice and training materials on
presentation of reports.
24th February 2009 4th Health IT Summer Showcase 9
Ultimate Goals of ProjectA. provide feedback to pathologists re the content of their
reports - are they including all the key information that:i. determines the patients prognosis andii. directs their managementThe key features that determine i & ii are:
a. Breslow thicknessb. mitotic rate
c. Clark level
d. ulceratione. margins (all of them)
24th February 2009 4th Health IT Summer Showcase 10
Ultimate GoalsB. data extraction for
i. cancer registriesii. research
C. automated generation of synoptic reports from text reportsi. could be done when the pathologist has constructed the narrative report and included in the final report that is sent to the requesting clinician ii. performed at a later date
24th February 2009 4th Health IT Summer Showcase 11
BACKGROUND TO THE STUDY • Materials
– 477 histopathology reports– Photocopied– Scanned– OCRed– Spell checked/ proof read– Anonymised– Stored and maintained in a revision repository– Annotated by pathologist and linguists
24th February 2009 4th Health IT Summer Showcase 12
Annotation Discrepancies Involving Language
• Noun Phrase vs Verb phase usage– Arising in a dysplastic naevus (useful verb)– Patchy regression was seen (not useful)
• Interpretive Annotations– Early, Intermediate, Late changed to– TILS, Fibrosis, Loss of Rete Ridges
24th February 2009 4th Health IT Summer Showcase 13
Concept SetAssociated naevus (type) Neurotropism
Breslow thickness (mm) Other Pattern(s)
Clark level Predominant cell type(s)
Classification/Main Pattern Regression
Dermal mitoses (per mm2) Rete Ridges
Desmoplasia (% of dermal invasive tumour) Satellites
Diagnosis Site
Distance from tumour to deep margin (mm) Solar elastosis
Fibrosis TILS
Nearest lateral margin to dermal invasive component (mm)
Ulceration (mm)
Nearest lateral margin to in-situ component (mm) Vascular/lymphatic invasion
24th February 2009 4th Health IT Summer Showcase 14
Results of the Annotations • Most important Concepts accurately
identified• Linguists had better agreement than the
pathologists• Three codes were not reliable
– TILS, Fibrosis, Rete Ridges
• Reliable Computation of important elements achievable
24th February 2009 4th Health IT Summer Showcase 15
Comparative Inter-annotator Agreement Second Annotation Round
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Diagnosis
Site
Clark level Lateral MarginBreslow Thickness
Classification
MitosesUlceration
Associated naevusVascular invasion
TILs
RegressionCell TypeDeep MarginNeurotropismIn-situ Margin
Fibrosis
Solar ElastosisDesmoplasia
Satellites
Other PatternsRete RidgesEarly (TILs)Intermediate
Late (Fibrosis)
Concept Tag
Inter-annotator Agreement
P1 & L1
P1 & L2
P2 & L1
P2 & L2
Figure 3 - Comparative second round inter-annotator agreement, scaled by number of annotations.
24th February 2009 4th Health IT Summer Showcase 16
Tags annotated by Pathologists missed by Linguists
24th February 2009 4th Health IT Summer Showcase 17
Comparison of Gold Standard and Linguists
Concept Linguist 1 Linguist 2
Breslow Thickness
.97 .97
Mitotic Rate .95 .94
Clark level .97 .94
Ulceration .94 .98
Margins .92/.92/.81 .98/.93/.94
24th February 2009 4th Health IT Summer Showcase 18
Observations about the corpus contents
• Occasional inconsistencies between report body and conclusions, e.g. size = .1mm vs 1mm
• Highly variable standard of contents
• Lateral margins not well reported
24th February 2009 4th Health IT Summer Showcase 19
Results of simple extraction for a Structured Report
• Breslow Thickness simple classifier
24th February 2009 4th Health IT Summer Showcase 20
Summary of the Project Results • Reliable annotations can be made for all the important
concepts in melanoma pathology reports.• The prospect of building very reliable computational
aides for automatically generating structured reports are high.
• The most uncertain aspect of the study is to understand the smallest training set that is needed to build an effective structured report computational populator.
• This approach can be used ot identify what is needed in any structured report -
“the text tells more than the experts”
24th February 2009 4th Health IT Summer Showcase 21
Breast Cancer Synoptic Reports• 33 categories
• 120+ reports
• Information extraction
24th February 2009 4th Health IT Summer Showcase 22
24th February 2009 4th Health IT Summer Showcase 23
24th February 2009 4th Health IT Summer Showcase 24
Projects for 20091. Snomed CT subset for ICU
2. Snomed CT subset for ED
3. Multiple CIS for Aged Care demonstrated
4. Trauma CIS verified and tested
5. First version of IC Realtime Audit IS (ICRAIS)
6. Automatic post co-ordination of clinical notes
7. Lexical and morphological disambiguation of clinical notes
8. Automatic computation of structured reports for melanoma and breast cancer
9. Proven use of CLINIDAL for pathology & ICU CIS
10. Mapping SNOMED CT to ICD 10 AM for ICU notes
11. Design of Information Model for ICD 11 (WHO)
24th February 2009 4th Health IT Summer Showcase 25
Partners• Breast Cancer Institute• RPAH - ICU, ED, AP• SWAPS• Blacktown-Mt Druitt Hospital - Nursing &
Midwifery• QUPC• SEALS
• Concorde - ED• NEHTA