Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang
description
Transcript of Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang
![Page 1: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/1.jpg)
Capturing and Modeling Neuro-Radiological
Knowledge on a Community
Basis: The Head Injury Scenario.
Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker,
and Suisheng Tang
Data Mining DepartmentInstitute for Infocomm Research
Singapore
![Page 2: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/2.jpg)
Outline
• Motivation
• MiBank – Head Injury Database
• Ontology Development
• Collective Intelligence
• The “facebook” approach
• Medical Image Annotator
• Discussion and Conclusions
![Page 3: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/3.jpg)
Motivation• National Neurological Institute Singapore (NII) has 500+
head injury patients each year with Brain, Scalp, Skull, Internal bleeding requiring rapid diagnosis.
• Clinical radiology reports comprise of multiple series of Computed Tomography (CT) Images with unstructured text associated to images)
• Computationally a weak association between images and words, cannot retrieve similar images.
• Conceptually a tightly coupled association between Image and Diagnosis
• MiBank database of DICOM files, (http://dicom.i2r.a-star.edu.sg/pacsone/)
![Page 4: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/4.jpg)
Features of MiBank Browser
Search by category, patient, report, note, study
Annotation with free-text
Forum discussion
DICOM viewer
Image upload & download
![Page 5: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/5.jpg)
MiBank: Medical Image Databank: Head Injury
Web site: http://dicom.i2r.a-star.edu.sg/pacsone/
505 studies, 1775 series, 31561 images. Pass word protect, DICOM viewer, searchable
![Page 6: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/6.jpg)
Show me all cases who have skull fracture with acute subdural hematoma But do not have brain edema.
Possible query in MiBank
Show all cases who have skull fracture with midline shift and acute subdural hematoma But do not have brain edema.
Details in next page
Impossible query with no-
predefined terms
![Page 7: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/7.jpg)
Current Limitations of MiBank
A fracture of the right frontal bone.
Mild midline shift to the left is present.
An acute extradural hematoma, measuring
1.9 cm in maximal thickness, is noted.
A 1 cm thick acute subdural hematoma is
also present over the right cerebral
hemisphere.
Sample: Radiology Report
Can not query based on image features explicitly;
Can not associate the description in R-report to specific instance of an image.
Need to see all instances for bone fracture
![Page 8: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/8.jpg)
What do we want… What do we need?
• Retrieve patients with right midnight shifts of less than 3mm for whom there has been no reported haematoma
• Retrieve all images similar to this one
• Properly annotated data: images, radiology reports
• Meaningful associations between reports, images, and across images
• …an ontology ….
![Page 9: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/9.jpg)
Head Injurydatabase
(Relational)
Categorization
Indexing
Header InfoMining
OriginalDICOM
data
Web In
terface
Search Engine
Visualization
Reportdata
Text mining
Image retrieve
Statistic report
Customized online report
Discussion forum
Semantic Query
Ontology
![Page 10: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/10.jpg)
The Role of the Ontology
• Community defined controlled vocabulary for annotation of radiology images.
• Hierarchical descriptions of medical terms relevant to anatomy, pathology and head injury specific features found in medical images.
• Consensus model of head injury terminology generated through community engagement for knowledge reuse in medical information systems.
• Query model for semantic search
![Page 11: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/11.jpg)
Ontology Development
Garcia et al
![Page 12: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/12.jpg)
Ontology Development
P h a s e 1 P h a s e 2
![Page 13: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/13.jpg)
Text Processing / Baseline Ontology
“Plain scans were acquired. Note is made of the MRI dated 2/3/2004 and CT dated 18/2/2004.Evidence of previous left high parietal craniectomy noted. Hypodensity in the left parietal-occipital region is compatible with gliosis at site of previous surgery. A large left-sided scalp hematoma is seen. Underlying linear radiolucency in the left frontal bone was seen. This suggests an undisplaced fracture. Underlying acute subdural hematoma is seen with a maximal depth of 1.2 cm. Acute subarachnoid blood is also noted collecting mainly in the ipsilateral cerebral hemisphere, sylvian fissure as well as tentorium. There is diffuse cerebral edema. Mass-effect is seen with midline shift to the right, and developing hydrocephalus. Basal cisterns are effaced”.
FMAFMA Non FMANon FMA
FMA / Galen / R-report terms: FMA / Galen / R-report terms: anatomy, pathology, trauma, injury anatomy, pathology, trauma, injury
![Page 14: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/14.jpg)
Capturing Knowledge: Phase 1
• Requires excessive amount of time • Experts – easily bored – no short term
result. • Results in the creation of unstructured
knowledge stores that are difficult to reuse and maintain.
• Skimping on validation may include errors, omissions, inconsistencies & irrelevances
• Experts are not always capturing the evidence – rather explaining context
• Storing the knowledge that is not machine-readable
Not an easy task
• Inside expert’s head
• Difficult to describe
concepts and relations
• Difficult for non-
experts to understand.
Disadvantages
![Page 15: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/15.jpg)
Ontology Development
P h a s e 1 P h a s e 2
Maintenance
Evolution
![Page 16: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/16.jpg)
Capturing Knowledge: Phase 2
• Collective Knowledge Resources– intelligent collection?
• collaborative bookmarking, searching
– “database of intentions”• clicking, rating, tagging,
buying - Amazon
– what we all know but hadn’t got around to saying in public before
• blogs, wikis,
discussion lists -
• Knowledge Elicitation via Collective Intelligence– The capacity to provide
useful information based on human contributions which gets better as more people participate.
– Data Types• mix of structured,
machine-readable data and unstructured data from human input
![Page 17: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/17.jpg)
Retrieving images of the diving trip to Australia. Albert and Alex have to be in the photo.
Tags Make The Difference !
• The Premise: From unstructured and unrelated annotation to structured meaningful annotation
• Simple tagging it possible to derive meaningful associations
• Need to have a tool to gather knowledge that is directly linked to supporting evidence.
![Page 18: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/18.jpg)
Medical Image Annotator: MIA• Main challenge in medical image retrieval
is that it heavily depends on expert’s knowledge of data structures and annotation is poor. So the objective of MIA is knowledge capture.
• MIA is designed for medical image annotation and its users are domain experts who require a consistent vocabulary for annotation tasks, knowledge sharing and machine automation.
• User community consists of Radiologists, Neurosurgeons (specifically, NNI doctors). Medical students, junior doctors, image processing researchers.
• MIA is a designed to both facilitate the building of appropriate ontology by domain experts and effective maintenance and evolution of the ontology, given new use cases /images.
MIA User Interface
Our contribution: the use of WEB 2.0 technology to support knowledge capture, and the approach to community engagement in the development of the ontology; more concretely in the maintenance and evolution
![Page 19: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/19.jpg)
MIA: Platform Architecture
Server-side processors Client-side browser
Image
Database
OntologyViewer
.owlfile
Java script (DHTML)
AJAX
Owl Parser
Tree Constructor
OntologyEditor
Ontology & ImageManagement Console
Ontologies can be edited online: * add node * rename node * delete node
OWL files can be loaded dynamicallyOWL relational database OWL
• Users can keep their own version of ontology• Consolidated ontology will be generated based on community inputs.
Ajax to update ontologies on server side to provide dynamic content on a web page so no page-refresh, no re-loading
Easy to extend, any OWL file can be loaded
![Page 20: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/20.jpg)
Knowledge Capture in Action
![Page 21: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/21.jpg)
Knowledge Capture in Action
![Page 22: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/22.jpg)
Knowledge Capture in Action
![Page 23: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/23.jpg)
Medical Image Annotator: MIAAdvantages
• Fast and easy• Domain experts lead the
process• Always rooted in reality or
a medical use case • Maintenance and evolution
of the controlled vocabulary is assured.
• Excellent training for new doctors / radiologists
• Facilitates Data Mining of Radiology reports
![Page 24: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/24.jpg)
Ontology Evolution
• Different trainee and clinical doctors building ontologies with extensions on
different sub trees
• Consolidated ontology is currently manually curated
• Goal is automatically align & merge ontologies
![Page 25: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/25.jpg)
Query with the Head Injury Ontology
1. Simple ‘ontology-term’ assisted query• Search for images: based merely on simple
combination of ontology terms (and / or)• Form based interface linked to SQL Queires
2. Ontology reasoning (A-box)• Content navigation over R-reports using defined
object properties (Knowlegtor)• Use of subsumption and object properties
![Page 26: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/26.jpg)
Concepts
(27)
Roles
(38)
• BrainRegion • has_BrainRegion
Information
• Symptom • has_Symptoms
• DiseaseDiagnosis • has_DiseaseDiagnosis
• DiseaseStage • has_DiseaseStage
• ImageReport • has_ImageReport
• ImageView
• TextReport
• has_ImageView
• has_TextReport
• Intracranial
Hemorrhage
• has_Intracranial
Haemorrhage
• Intraventricular
Hemorrhage
• has_Intrventricular
Haemorrhage
Head Injury Ontology
![Page 27: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/27.jpg)
Find patient records for ‘Fracture’
![Page 28: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/28.jpg)
Discussion and Conclusions• Medical images should be better annotated in order to facilitate
information retrieval
• Collective knowledge is real… “FAQ-o-Sphere”
• Controlled vocabularies (CVs) and/or ontologies are being developed by communities
• Simple tagging combined with knowledge elicitation methods supports ontology development
• Collective knowledge capture requires dedicated infrastructure that supports specific tasks
• Querability can be improved through the use of explicit tags and CVs/ontologies
![Page 29: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/29.jpg)
Social Web Social + Semantic Web
• How to get knowledge from all those intelligent people on the Internet
• How to give everyone the benefit of everyone else’s experience
• How to leverage and contribute to the ecosystem that has created today’s web.
Challenges for the Community
Life ScienceLife Science
![Page 30: Alexander Garcia, Zhuo Zhang, Menaka Rajapakse, Christopher J. O. Baker , and Suisheng Tang](https://reader036.fdocuments.in/reader036/viewer/2022062314/56813a2c550346895da211f4/html5/thumbnails/30.jpg)
Acknowledgments • Bonarges Aleman-Meza – Social Web• Tom Gruber - Semantic-Social Web• MIA Developers - Zhang Zhuo and Menaka
Rajapakse• Suisheng Tang M.D. and Project PI, - Coordinator
of domain experts and builder of baseline ontology• Tchoyoson Lim – Radiologist NNI (National
Neuroscience Institute, Singapore)