Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval...
-
date post
21-Dec-2015 -
Category
Documents
-
view
221 -
download
2
Transcript of Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval...
![Page 1: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/1.jpg)
Knowledge-Based Interpretation, Visualization,
and Exploration of Time-Oriented Medical Data
Yuval Shahar, M.D., Ph.D.
Medical Informatics CenterInformation Systems Engineering
Ben Gurion UniversityBeer Sheva, Israel
AndDepartments of Medicine and Computer Science
Stanford Medical InformaticsStanford UniversityStanford, CA, USA
![Page 2: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/2.jpg)
Time in Natural Language
From—
“Mr. Jones was alive after Dr. Smith operated on him”
Does it follow that—
“Dr. Smith operated on Mr. Jones before Mr. Jones was alive?”
Is Before the inverse of After?
![Page 3: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/3.jpg)
Timing is Everything :Applications of Temporal Reasoning
• Natural-language processing (e.g., medical record understanding)
• Planning (e.g., robot planning, therapy planning)
• Causal reasoning (e.g., diagnosis)
• Archeology (e.g., seriation)
• Psychology (e.g., developmental beahvioral psychology)
• Scheduling (e.g., optimal ordering)
• Circuit design (e.g., sequential circuits)
• Software design (e.g., parallel processing, communication, verification)
• Other, not necessarily time-oriented, domains where interval algebra is useful, such as molecular biology (e.g., arrangement of DNA segments along a linear DNA chain) and evaluation of spatiotemporal traffic-control patterns
![Page 4: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/4.jpg)
Allen's Temporal Logic (1981–1984)• Only temporal intervals - no instantaneous events
• 13 basic (binary) interval relations (b,a,eq,o,oi,s,si,f,fi,d,di,m,mi)
and transitivity relations between them
• Properties hold over every subinterval of an interval
—> Holds(p, T) e.g., "Patient1's skin was blue throughout sunday"
• Events hold only over an interval and not over any subinterval of it
—> Occurs(e, T) e.g., "patient2 broke a leg at 5pm"
• Processes hold over some subintervals of the interval they occur in
—> Occuring(p, T) e.g., "patient3 is chasing the nurse"
![Page 5: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/5.jpg)
Allen’s 13 Temporal RelationsA
B
A
B
A
B
A
B
A
B
A
B
A
B
A is EQUAL TO B
A is BEFORE B
B is EQUAL TO A
B is AFTER A
A MEETS B
B is MET BY A
A OVERLAPS B
B is OVERLAPPED BY A
A STARTS B
B is STARTED BY A
A FINISHES B
B is FINISHED BY A
A is DURING B
B CONTAINS A
![Page 6: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/6.jpg)
The Temporal-Abstraction Task
• Input: time-stamped clinical data and relevant events (interventions)
• Output: interval-based abstractions
• Identifies past and present trends and states
• Supports decisions based on temporal patterns, such as: “modify therapy if the patient has a second episode of Grade II bone-marrow toxicity lasting more than 3 weeks”
• Focuses on interpretation, rather than on forecasting
![Page 7: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/7.jpg)
Temporal Abstraction:The Bone-Marrow Transplantation
Domain
.
•
0 40020010050
•
1000
2000( )
100K
150K
( )
•••
• • • ••
• •
•••
Granu-locytecounts
• • •
•
Time (days)
Plateletcounts
PAZ protocol
M[0] M[1] M[2] M[3] M[1] M[0]
BMT
Expected CGVHD
![Page 8: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/8.jpg)
Temporal Abstractions: A Graphical View
![Page 9: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/9.jpg)
Uses of Temporal Abstractions
• Therapy planning and patient monitoring. E.g., the EON project (a modular architecture to support guideline-based care)
• Creating high-level summaries of time-oriented medical records
• Supporting an explanation module for a medical DSS
• Representing goals and policies of therapy plans and guidelines for quality assessment purposes (at runtime and retrospectively). E.g., the Asgaard project: Intentions of guideline designers with respect to both process and outcomes are captured as temporal patterns to be achieved or avoided.
• Visualization of time-oriented clinical data: the KNAVE project
![Page 10: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/10.jpg)
The Temporal-Abstraction Ontology
• Events (insulin therapy) - part-of, is-a relations
• Parameters (hemoglobin values and abstractions) - abstracted-into, is-a relations
• Abstraction goals (therapy of diabetes patients) - is-a relations
• Interpretation contexts (effect of regular insulin) - subcontext, is-a relations
• Interpretation contexts are induced by other entities and can have any temporal relationship to the inducing entity
![Page 11: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/11.jpg)
Temporal-Abstraction Output Types
• State abstractions (LOW, HIGH)• Gradient abstractions (INC, DEC)• Rate Abstractions (SLOW, FAST) • Pattern Abstractions (CRESCENDO)
- Linear patterns- Periodic patterns
![Page 12: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/12.jpg)
Temperature
Hemoglobin Level
Linear Component
Week 2 Week 3Week 1
Anemia
Fever Fever
Anemia Anemia
FeverFever
Anemia
Fever
Linear ComponentLinear Component Linear Component
Periodic Pattern
Abstraction of Periodic Temporal Patterns
![Page 13: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/13.jpg)
Temporal-Abstraction Knowledge Types
• Structural (e.g., part-of, is-a relations) - mainly declarative/relational
• Classification (e.g., value ranges; patterns) - mainly functional
• Temporal-semantic (e.g., “concatenable” property) - mainly logical • Temporal-dynamic (e.g., interpolation functions) - mainly probabilistic
![Page 14: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/14.jpg)
The RÉSUMÉ System Architecture
Temporal-abstraction mechanisms
Temporal fact baseE v e n t s
C o n t e x t s
A b s t r a c t e d i n t e r v a l s
P r i m i t i v e d a t a •
Domain knowledge base
Event ontology
Parameter ontology
Primitive data
Events
••
Context ontology
External patient database
+ +
+
•+
![Page 15: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/15.jpg)
Test Domains for the RÉSUMÉ System
• Medical domains:– Guideline-based care
• AIDS therapy
• Oncology
– Monitoring of children’s growth
– Therapy of insulin-dependent diabetes patients
• Non-medical domains:– Evaluation of traffic-controllers actions
– summarization of meteorological data
![Page 16: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/16.jpg)
Acquisition of Temporal-Abstraction Knowledge
![Page 17: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/17.jpg)
Temporal Reasoning and Temporal Maintenance
• Temporal reasoning supports inference tasks involving time-oriented data; often connected with artificial-intelligence methods
• Temporal data maintenance deals with storage and retrieval of data that has multiple temporal dimensions; often connected with database systems
• Both require temporal data modelling
Clinicaldecision-supportapplication
TM TR DB
![Page 18: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/18.jpg)
Tzolkin: A Temporal-Mediation Architecture
or:Combining Temporal Reasoning and Temporal
Maintenance
DB
Temporal-mediation controller
RÉSUMÉ (TR)
Chronus (TM)
Clinicaldecision-supportapplication
![Page 19: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/19.jpg)
Knowledge-Based Visualization andExploration of Time-Oriented Medical Data:
Desiderata
• Interactive composition of (temporal-abstraction) queries
• Visualization of query results
• Exploration of multiple levels of temporal abstractions
• The semantics of the query, visualization and exploration operators should be domain independent, but should use the terms and relations specific to each (e.g., medical) domain
![Page 20: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/20.jpg)
The KNAVE Project(Knowledge-Based Navigation of Abstractions for Visualization and
Explanation)
• A conceptual and computational framework for temporal abstraction, visualization, and exploration
• Capitalizes on existing components (RÉSUMÉ, temporal database mediator, KA tool, domain-knowledge server)
• The exploration operators reuse (and are defined by) the domain’s temporal-abstraction ontology
• Introduces new graphical and computational modules
![Page 21: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/21.jpg)
The KNAVE ArchitectureTemporal Mediator
DB
Visualization andexploration module Computational
Module
GraphicalInterface
Domain-knowledge Server
End user
Expertphysician
KATool
Résumé
Chronus
Controller
KB
Ontology server
![Page 22: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/22.jpg)
Beginning a Visualization Session: A Temporal-Abstraction Query
![Page 23: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/23.jpg)
The Browsing and Exploration Interface
![Page 24: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/24.jpg)
Semantic Exploration Operators• Motion across semantic links in the domain’s knowledge base;
in particular, relations (and their inverse) such as: - part-of - is-a - abstracted-from - subcontext • Motion across abstraction types: state, gradient, rate, pattern
• Application of aggregation operators such as mean and distribution
• Dynamic change of temporal-granularity (e.g., from days to months) changes the display, using domain-specific aggregation knowledge
• Explanation by display of relevant knowledge, or through “What-if” queries, which allow hypothetical assertion or retraction of data or knowledge and examination of resultant patterns
![Page 25: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/25.jpg)
An Abstracted-From Exploration Result
![Page 26: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/26.jpg)
A Statistical-Query Example
![Page 27: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/27.jpg)
Responding to an Explanation Query (“How”):
A Bone-Marrow–toxicity Classification Table
![Page 28: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/28.jpg)
The Preliminary Evaluation Study
• Developmental assessment of the prototype
• Seven users with varying medical/computer use backgrounds
• Each user given a 10 minute introduction to the KNAVE system
• A single electronic patient file constructed from several cases in the domains of AIDS and bone-marrow transplantation
• Each user asked to perform three tasks (a complex temporal query, a context-sensitive abstraction, and a statistical query)
• Qualitative impression and quantitative (time) measures noted
![Page 29: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/29.jpg)
The Preliminary-Evaluation Results
• All users answered all queries within 3 minutes; 6 of 7 users completed all three tasks within 90 seconds
• All users expressed enthusiasm and found the interface useful
• Striking redundancy noted in use of interface: At least four different paths were found to the same answers, and five different patterns of use of the exploration operators
• Difficult to compare to manual tools, since these do not support any automated abstraction or explanation of such
![Page 30: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/30.jpg)
KNAVE: Current State and Future Directions
• Basic prototype in Visual Basic; Java implementation under way
• Collaboration with an industrial company to create a web-based version
• Current research issues:
– Implementation of temporal-granularity semantic zoom
– Runtime linear and periodic pattern queries
– Semantics and implementation of distributed What-If queries, which modify either the knowledge or the data at runtime and examine the effect of the result on the displayed patterns
– Enhancement of RÉSUMÉ and the KA tool as needed, including integration with statistical tools
– Future link to a text summarization module
![Page 31: Knowledge-Based Interpretation, Visualization, and Exploration of Time-Oriented Medical Data Yuval Shahar, M.D., Ph.D. Medical Informatics Center Information.](https://reader036.fdocuments.in/reader036/viewer/2022062516/56649d645503460f94a46f43/html5/thumbnails/31.jpg)
Temporal-Abstraction and Visualization:Conclusions
• Temporal abstraction of time-oriented data can employ reusable domain-independent computational mechanisms that rely on access to a domain-specific temporal-abstraction ontology
• Temporal abstraction is useful for planning, monitoring, data summarization and visualization, explanation and critiquing
• Interactive query, visualization of, and exploration requires runtime access to the domain’s temporal-abstraction ontology
• The visualization and exploration semantics can be specific to the temporal-abstraction task, but not to the domain