A Semantics-based Approach to Machine Perception
-
Upload
knoesis-center-wright-state-university -
Category
Education
-
view
980 -
download
0
description
Transcript of A Semantics-based Approach to Machine Perception
![Page 1: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/1.jpg)
A SEMANTICS-BASED APPROACHTO MACHINE PERCEPTION
Cory Andrew Henson
August 27, 2013
1
Committee: Amit Sheth (advisor)
Krishnaprasad Thirunarayan John Gallagher
Payam Barnaghi Satya Sahoo
Ph.D. Dissertation Defense
Wright State UniversityDept. of Computer Science and Engineering
![Page 2: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/2.jpg)
2
![Page 3: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/3.jpg)
3
Thesis
Machine perception can be formalized using semantic web technologies in order to derive abstractions from sensor data using background knowledge on the Web, and efficiently executed on resource-constrained devices.
![Page 4: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/4.jpg)
4
3 primary issues to be addressed
Annotation of sensor data
SemanticSensor
Web
SemanticPerception
Intelligence
at the Edge
Interpretation of sensor data
Efficient execution onresource-constrained devices1 2 3
![Page 5: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/5.jpg)
5
lives in
has pet
is ahas petPerson Animal
Concrete Facts Resource Description Framework
Semantic Web(according to Farside)
General Knowledge Web Ontology Language
“Now! – That should clear up a few things around here!”
is a
![Page 6: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/6.jpg)
Government
Media
Publications
Life SciencesGeographic
Semantic Web
6
![Page 7: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/7.jpg)
7http://www.opengeospatial.org/projects/groups/sensorwebdwg
Semantic Sensor Web
![Page 8: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/8.jpg)
8
Sensor systems are too often stovepiped
![Page 9: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/9.jpg)
9
With freedom comes responsibility1. discovery, access, and
search2. integration and
interpretation
We want to set this data free
![Page 10: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/10.jpg)
10
OGC Sensor Web Enablement (SWE)
![Page 11: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/11.jpg)
11
With freedom comes responsibility1. discovery, access, and
search2. integration and
interpretation
We want to set this data free
![Page 12: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/12.jpg)
12
RDF OWL
How are machines supposed to integrate and interpret sensor data?
Semantic Sensor Networks (SSN)
![Page 13: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/13.jpg)
13
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
![Page 14: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/14.jpg)
14
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
![Page 15: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/15.jpg)
15
W3C Semantic Sensor Network Ontology
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
![Page 16: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/16.jpg)
16
Semantic Annotation of SWE
Lefort, L., Henson, C., Taylor, K., Barnaghi, P., Compton, M., Corcho, O., Garcia-Castro, R., Graybeal, J., Herzog, A., Janowicz, K., Neuhaus, H., Nikolov, A., and Page, K.: Semantic Sensor Network XG Final Report, W3C Incubator Group Report (2011).
![Page 17: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/17.jpg)
Semantic Sensor Observation Service (SemSOS)
Cory Henson, Josh Pschorr,Amit Sheth, Krishnaprasad Thirunarayan, SemSOS: Semantic Sensor Observation Service, In Proceedings of the 2009 International Symposium on Collaborative Technologies and Systems (CTS 2009), Baltimore, MD, May 18-22, 2009. 17
![Page 18: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/18.jpg)
Semantic Sensor Observation Service (SemSOS)
Joshua Pschorr, Cory Henson, Harshal Patni, and Amit P. Sheth. Sensor Discovery on Linked Data. Kno.e.sis Center Technical Report 2010.
18
![Page 19: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/19.jpg)
19
3 primary issues to be addressed
Annotation of sensor data
SemanticSensor
Web
SemanticPerception
Intelligence
at the Edge
Interpretation of sensor data
Efficient execution onresource-constrained devices1 2 3
![Page 20: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/20.jpg)
20
Semantic Perception
• The role of perception is to transform raw sensory data into a meaningful and correct representation of the external world.
• The systematic automation of this ability is the focus of machine perception.
• For correct interpretation of this representation, we need a formal account of how this is done.
![Page 21: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/21.jpg)
21
What can we learn from cognitive models of
perception?
People are good atmaking sense
of sensory input
![Page 22: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/22.jpg)
22
Perception is an active, cyclical process of exploration and interpretation.
The perception cycle is driven by prior knowledge, in order to generate and test hypotheses.
Some observations are more informative than others (in order to effectively test hypotheses*).
Ulric Neisser
Richard GregoryKenneth Norwich
1970’s 1980’s 1990’s
Cognitive Models of Perception
* Applies to machine perception within Intellego, NOT human perception.
![Page 23: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/23.jpg)
23
ObservedPropertie
s
PerceivedFeatures
Background knowledge
Explanation
Ontology of Perception
Focus
An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
![Page 24: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/24.jpg)
Example: Medical diagnosis as perception
24
![Page 25: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/25.jpg)
Proactive,Preventative
Healthcare
25
![Page 26: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/26.jpg)
26
The Patient of the Future
MIT Technology Review, 2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/
![Page 27: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/27.jpg)
Digital Doctor
Let’s provide people with the tools needed to monitor and
manage their own health
http://worldofdtcmarketing.com/mobile-health-apps-a-new-opportunity-for-healthcare-marketers/mobile-healthcare-marketing-trends/ 27
![Page 28: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/28.jpg)
Medical/healthcare expert systems have been around for a long time
28
![Page 29: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/29.jpg)
1. Ubiquitous Sensing 2. Always-on Computing3. Knowledge on the
Web
3 recent developments have changed the technological landscape …
29
![Page 30: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/30.jpg)
30
Making sense of sensor data with
![Page 31: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/31.jpg)
DATAsensor observations
KNOWLEDGEsituation awareness useful
for decision making
Primary challenge is to bridge the gap between data and knowledge
31
![Page 32: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/32.jpg)
SSNOntology
2 Interpreted data(deductive)[in OWL] e.g., threshold
1 Annotated Data[in RDF]e.g., label
0 Raw Data[in TEXT]e.g., number
Levels of Abstraction
3 Interpreted data (abductive)[in OWL]e.g., diagnosis
Intellego
“150”
Systolic blood pressure of 150 mmHg
ElevatedBlood
Pressure
Hyperthyroidism
less
use
ful …
…
mor
e us
eful
……
32
![Page 33: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/33.jpg)
33
ObservedPropertie
s
PerceivedFeatures
Background knowledgeon the Web
Low-level observed properties suggest explanatory hypotheses through abduction
Explanation
Focus
Ontology of Perception
An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
![Page 34: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/34.jpg)
34
Semantics of Explanation
Abduction – or, inference to the best EXPLANATION
Task• Given background knowledge of the environment (SIGMA), and• given a set of sensor observation data (RHO),• find a consistent explanation of the situation (DELTA)
Backgroundknowledge Features
(objects/events)in the world
Sensor observation
data
![Page 35: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/35.jpg)
35
Semantics of Explanation
Background knowledge is represented as a causal network between features (objects or events) in the world and the sensor observations they give rise to.
![Page 36: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/36.jpg)
36
Semantics of Explanation
Finding the sweet spot between abduction and OWL
• Simulation of Parsimonious Covering Theory in OWL-DL (using the single-feature assumption*)
* An explanation must be a single feature which
accounts forall observed properties
Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012)
Theorem: Given a PCT problem P and its translation o(P) into OWL, Δ = {e} is a PCT explanation if and only if ExplanatoryFeature(e) is deduced by an OWL-DL reasoner — that is, if and only if o(P) ⊧ ExplanatoryFeature(e).
![Page 37: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/37.jpg)
37
Finding the Sweet Spot
minimizeexplanations
degrade gracefullyw/ incomplete info
decidable
web reasoning
Abductive Logic (e.g., PCT)high complexity
Deductive Logic (e.g., OWL)(relatively) low complexity
Explanation(in Intellego)
![Page 38: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/38.jpg)
38
Explanatory Feature: A feature is explanatory w.r.t. a set of observed properties if it causes each property in the set.
ExplanatoryFeature ≡ isPropertyOf∃ —.{p1} … isPropertyOf⊓ ⊓ ∃ —.{pn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Observed Property Explanatory Feature
Semantics of Explanation
![Page 39: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/39.jpg)
39
ObservedPropertie
s
PerceivedFeatures
Background knowledgeon the Web
Hypotheses imply the informational value of future observations through deduction
Explanation
Focus
Ontology of Perception
An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)
![Page 40: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/40.jpg)
40
Universe of observable properties
Semantics of Focus
To predict which future observations have informational value, find those observable properties that can discriminate between the set of hypotheses.
ExpectedProperties
Not-applicableProperties
Discriminating
Properties
![Page 41: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/41.jpg)
41
Expected Property: A property is expected w.r.t. a set of features if it is caused by each feature in the set.
ExpectedProperty ≡ isPropertyOf.{f∃ 1} … isPropertyOf.{f⊓ ⊓ ∃ n}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Expected Property Explanatory Feature
Semantics of Focus
![Page 42: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/42.jpg)
42
Not Applicable Property: A property is not-applicable w.r.t. a set of features if it is not caused by any feature in the set.
NotApplicableProperty ≡ ¬∃isPropertyOf.{f1} … ¬⊓ ⊓ ∃isPropertyOf.{fn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Not Applicable Property Explanatory Feature
Semantics of Focus
![Page 43: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/43.jpg)
43
Discriminating Property: A property is discriminating w.r.t. a set of features if it is neither expected nor not-applicable.
DiscriminatingProperty ≡ ¬ExpectedProperty ¬NotApplicableProperty⊓
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Discriminating Property
Explanatory Feature
Semantics of Focus
![Page 44: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/44.jpg)
44
Off-the-shelf OWL-DL reasoners are too resource intensive in terms of both memory and time
• Runs out of resources with background knowledge >> 20 nodes
• Asymptotic complexity: O(n3)
O(n3) < x < O(n4)
Semantic perception on resource-constrained devices
![Page 45: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/45.jpg)
45
3 primary issues to be addressed
Annotation of sensor data
SemanticSensor
Web
SemanticPerception
Intelligence
at the Edge
Interpretation of sensor data
Efficient execution onresource-constrained devices1 2 3
![Page 46: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/46.jpg)
Internet of Things
46http://www.idgconnect.com/blog-abstract/900/the-internet-things-breaking-down-barriers-connected-world
![Page 47: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/47.jpg)
47http://share.cisco.com/internet-of-things.html
![Page 48: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/48.jpg)
48
Basis Watch• Heart Rate Monitor• Accelerometer• Skin Temperature• Galvanic Skin Response
![Page 49: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/49.jpg)
Homo Digitus
49
How do we make sense of this data … and do it efficiently and at scale?
![Page 50: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/50.jpg)
50
Approach 1: Send all sensor observations to the cloud for processing
Approach 2: downscale semantic processing so that each device is capable of machine perception
Intelligence at the Edge
![Page 51: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/51.jpg)
51
Use bit vector encodings and their operations to encode background knowledge and execute perceptual inference
Efficient execution of semantic perception
0110001110001110110011100101011001110101
OWL-DL
An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (ISWC, 2012)
![Page 52: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/52.jpg)
52
lift
lower
Translate background knowledge, observations, and explanations between Semantic Web and bit vector representation
Lifting and lowering of knowledge
![Page 53: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/53.jpg)
53
Efficient execution of semantic perception
Bit vector algorithms are provably equivalent to the OWL inference (i.e., semantics preserving)
Intuition: discover and dismiss those features that cannot explain the set of observed properties.
Intuition: discover and assemble those properties that discriminate between the explanatory features
![Page 54: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/54.jpg)
54
bp 1
cs 0
pa 1
HN HM PE
bp 1 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 1
HN HM PE
1 1 1
1 1 0
AND =>
AND
1 1 1
ObservedProperty Prior Knowledge
PreviousExplanatory Feature
CurrentExplanatory Feature
=>
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and dismiss those features that cannot explain the set of observed properties.
Explanation: efficient algorithm
![Page 55: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/55.jpg)
55
bp 1
cs 0
pa 1
HN HM PE
bp 0 1 1
cs 0 1 0
pa 1 1 0
HN HM PE
1 1 0
HN HM PE
0 1 0AND => 0 1 0
ObservedProperty Prior Knowledge
PreviousExplanatory Feature
CurrentExplanatory Feature
=
INTUITION: The strategy employed relies on the use of the bit vector AND operation to discover and assemble those properties that discriminate between the explanatory features
bp 0
cs 0
pa 0
Discriminating
Property
1 1 0
… expected?
0 0 0… not-applicable?ZERO Bit Vector
0 1 0
=
=> FALSE
=> FALSE
1
Is the propertydiscriminating?
Focus: efficient algorithm
![Page 56: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/56.jpg)
56
O(n3) < x < O(n4) O(n)
Evaluation on a mobile device
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes• Time reduced from minutes to milliseconds• Complexity growth reduced from polynomial to
linear
![Page 57: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/57.jpg)
57
Technical contributions in a nutshell
1. Semantic Sensor Web: Developed technologies for the semantic annotation of sensor data on the Web
- Semantic Sensor Web (IEEE Internet Computing, 2008) – 276 citations (as of Aug. 2013)- SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and
Systems, 2009)- Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011)
2. Semantic Perception: Designed a declarative specification of perception, capable of utilizing an off-the-shelf OWL-DL reasoner
- An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology Journal, 2011)
- Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing: Special Issue on Context-Aware Computing, 2012) – most downloaded paid paper in IEEE-IC 2012
3. Intelligence at the Edge: Implemented efficient algorithms for executing perceptual inference on resource-constrained devices
- An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference, 2012)
![Page 58: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/58.jpg)
58
W3C Reports1. Semantic Sensor Network XG Final Report (W3C Incubator Group Report, 2011)
Journal Publications2. Physical-Cyber-Social Computing: An Early 21st Century Approach (IEEE Intelligent Systems, 2013)3. Semantic Perception: Converting Sensory Observations to Abstractions (IEEE Internet Computing, 2012)4. Semantics for the Internet of Things: Early Progress and Back to the Future (International Journal on Semantic Web and Information Systems, 2012)5. The SSN ontology of the W3C semantic sensor network incubator group (Web Semantics: Science, Services and Agents on the World Wide Web, 2012)6. An Ontological Approach to Focusing Attention and Enhancing Machine Perception on the Web (Applied Ontology, 2011)7. Semantic Sensor Web (IEEE Internet Computing, 2008)
Conference Publications8. An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices (International Semantic Web Conference,
2012)9. Computing Perception from Sensor Data (IEEE Sensors Conference, 2012)10. SemSOS: Semantic Sensor Observation Service (International Symposium on Collaborative Technologies and Systems, 2009)11. Situation Awareness via Abductive Reasoning for Semantic Sensor Data: A Preliminary Report (International Symposium on Collaborative
Technologies and Systems, 2009).
Workshop Publications12. SECURE: Semantics Empowered Rescue Environment (International Workshop on Semantic Sensor Networks, 2011) 13. Representation of Parsimonious Covering Theory in OWL-DL (International Workshop on OWL: Experiences and Directions, 2011)14. Provenance Aware Linked Sensor Data (Workshop on Trust and Privacy on the Social and Semantic Web, 2010)15. Linked Sensor Data (International Symposium on Collaborative Technologies and Systems, 2010)16. A Survey of the Semantic Specification of Sensors (International Workshop on Semantic Sensor Networks, 2009)17. An Ontological Representation of Time Series Observations on the Semantic Sensor Web (International Workshop on the Semantic Sensor Web)18. Video on the Semantic Sensor Web (W3C Video on the Web Workshop, 2007)
Relevant Publications
![Page 59: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/59.jpg)
59
Application
Proactive, preventative healthcare
![Page 60: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/60.jpg)
Heart disease is a critical issue
~815,000 (2011)
http://millionhearts.hhs.gov/abouthds/cost-consequences.html 60
![Page 61: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/61.jpg)
Acute Decompensated Heart Failure (ADHF)
• Affects nearly 6 million people (in
U.S.)
• 555,000 new cases are diagnosed
each year
61U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
![Page 62: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/62.jpg)
• 4.8 million hospitalizations per year
• 50% are readmitted within 6
months
• 25% are readmitted within 30 days
• 70% due to worsening conditions
• Costing $17 billion per year
ADHF hospital readmission rates are too high
62U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
![Page 63: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/63.jpg)
Congress has incentivized hospitals to lower readmission rates
63U.S. Department of Health & Human Services. (2011). Hospital Compare. http://www.hospitalcompare.hhs.gov (Accessed on February 19, 2012).
![Page 64: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/64.jpg)
Current state-of-the-art
64
![Page 65: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/65.jpg)
Score (0: Not at all, 1: A little, 2: A great deal, 3: Extremely)
Heart Failure Somatic Awareness Scale (HFSAS)
Current state-of-the-art
65
![Page 66: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/66.jpg)
kHealth – knowledge-enabled healthcare
Approach: • Use semantic perception inference• with data from cardio-related sensors• and curated medical background knowledge on
the Web
1. to monitor and abstract health conditions2. to ask the patient contextually relevant
questions
66
![Page 67: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/67.jpg)
Cardiology Background Knowledge
• Symptoms: 284
• Disorders: 173
• Causal Relations: 1944
Unified Medical Language System
Causal Network
67
![Page 68: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/68.jpg)
kHealth Kit
Weight Scale
Heart Rate Monitor
Blood PressureMonitor
68
Sensors
Android Device (w/ kHealth App)
Total cost: < $500
![Page 69: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/69.jpg)
69
Explanation in kHealth
• Abnormal heart rate• High blood pressure
• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic Shock
Observed Property Explanatory Feature
via Bluetooth
![Page 70: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/70.jpg)
70
Focus in kHealth
Are you feeling lightheaded?
Are you have trouble taking deep breaths?
yes
yes
• Abnormal heart rate• High blood pressure• Lightheaded• Trouble breathing
• Panic Disorder• Hypoglycemia• Hyperthyroidism• Heart Attack• Septic Shock
Contextually dependent questioning based on prior observations(from 284 possible questions)
Observed Property Explanatory Feature
![Page 71: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/71.jpg)
71
Evaluation of kHealth
Evaluate the ability to discriminate between sets of potential disorders using:
1. HFSAS/WANDA’s restricted set of observable symptoms (12)
2. kHealth’s more comprehensive set of observable symptoms (284)
![Page 72: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/72.jpg)
72
Evaluation of kHealth
Evaluation Metrics:1. Efficiency: How many observations (or questions) required to
minimize the set of explanations?2. Specificity: How specific is the resulting minimum set of
explanations (i.e., how many explanatory disorders in the set)?
Explanatory Disorders
(computed by Intellego)
Actual Disorder(extracted from EMR)
Possible Disorders(derived from cardiology
KB)
![Page 73: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/73.jpg)
73
Evaluation of kHealth – Early Results
HFSAS/WANDAEfficiency: ~7.45 (# questions asked)Specificity: ~11.95* (# minimum
explanations)
* Converged to 1 explanation 20% of the time
• 496 EMRs• ~3.2 diagnosed
disorders per EMR
• 173 possible disorders
The approach utilized by kHealth is
more efficient and more specificthan HFSAS/WANDA.
kHealthEfficiency: ~7.28 (# questions asked)Specificity: 1 (# minimum explanations)
Explanatory Disorders
(computed by Intellego)
Actual Disorder(extracted from EMR)
Possible Disorders(derived from cardiology
KB)
![Page 74: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/74.jpg)
Pre-clinical usability trial
Dr. William Abraham, M.D.Director of Cardiovascular
Medicine
74
![Page 75: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/75.jpg)
Sensingand
Perception
HealthCare
Academic Standards
Org.
Industry Government
ResearchCollaborators
by Research Topicand Organization Type
75
![Page 76: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/76.jpg)
Special Thanks to AFRL and DAGSI
76
AFRL/DAGSI Research Topic SN08-8: Architectures for Secure Semantic Sensor Networks for
Multi-Layered Sensing
![Page 77: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/77.jpg)
77
![Page 78: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/78.jpg)
Semantic Sensor Web Team
78
![Page 79: A Semantics-based Approach to Machine Perception](https://reader033.fdocuments.in/reader033/viewer/2022060107/554aee97b4c9059f798b4865/html5/thumbnails/79.jpg)
A SEMANTICS-BASED APPROACHTO MACHINE PERCEPTION
Cory Andrew Henson
August 27, 2013
79
Thank you.
For additional information visit: http://knoesis.org/researchers/cory