SBIR Topic A04-105: An Ontologically-Based Data Fusion Model

Chris Matheus & Mitch Kokar

Versatile Information Systems, Inc.

cmatheus, mkokar@vistology.com

www.vistology.com

Outline

• JDL Model• Motivation, objectives, requirements• Meta-modeling framework• IFRM – top level• IFRM – first step refinements• Ontologies and fusion• How many ontologies can be used?• Commercialization – an idea• Tasks• Backup slides

SOURCE PRE - PROCESSING

LEVEL ONE PROCESSING

OBJECT REFINEMENT

LEVEL TWO PROCESSING

SITUATION REFINEMENT

LEVEL THREE PROCESSING

THREAT

REFINEMENT

LEVEL FOUR PROCESSING

PROCESS REFINEMENT

DATA FUSION DOMAIN

DATA BASE MANAGEMENT SYSTEM

FUSION DATABASE

SUPPORT DATABASE

SOURCES

LOCALDISTRIBUTEDNATIONAL

INTEL

EW

SONAR

RADAR . . .

DATA BASES

HUMAN COMPUTER

INTERACTION

USERS

WA - SOT assessment for Multi - source/sensor capability:

Mature / AvailableLab Demos / Not FieldedResearch Required

* From JDL/DFG Data Fusion Model

JDL Model

Fusion: JDL Model

Motivation

• JDL has seen a lot of attention

• But mainly Level 1 has been tested

• Recent emphasis on higher-level fusion made JDL insufficient

• Moreover, JDL has been misinterpreted as a data flow model

• Still needs a process model– Connected by “bus” – too flexible– Layered – too restrictive

Reference Model - Impacts

• Improve development efficiency• Enable application portability and

scalability• Ease application adaptability• Improve system interoperability (NCW!)• Improve user productivity and portability• Promote vendor independence• Improve security• Reduce life-cycle cost

Phase I Objectives

• Analyze reference models, identify useful features

• Propose adding process model to JDL• Formalize proposed model• Propose model evaluation approach and tools• Select application for validation in Phase II• Disseminate findings (papers, presentations)• Prepare an RFP for the OMG

Model Requirements

• Be descriptive – serve the fusion community• Capture data, functions, processes• Represent fusion processes – allow comparison of

systems before they are built• Capture metrics of performance• Ontologically based – formal, computer processable

semantics• Formal specification of the model itself• Have a place for capturing human-in-the-loop and

user models• Have associated software tools• Compatibility with current models (as much as

possible, but not more)

Meta-Modeling Framework (MMF)

MetaModel

Model

Objects Objects of interest in the domain (instances)

Classes of objects and relations among classes(Need modeling elements to define models)

Definition of modeling elements

MMF – Software Engineering

MetaModel

Model

Objects new Employee(“John”)new Employer(“IBM”)

Class, Association, …UML

ClassDiagrams

JavaObjects

Employee:Class, employedBy:Association

MMF – Semantic Web

MetaModel

Model

Objects <Employee John /><Employer IBM />

Class, Property, …OWL

Ontology

Annotation(markup)

<Class Employee /><Property employedBy />

MMF – Information Fusion

MetaModel

Model

Objects track51:Track1, tankA1:TankB,

IFRM Terms (Class, Property, Track,Object, Situation)

IFRM

Model ofFusion System

Fusion System(run time)

System model (Track1:Track, O1:Object,TankB:Tank)

Three Views

Process

Function

0..*

1

Performs

Product

0..*

1

IsResponsibleFor

2..*0..*Uses

0..*0..*Produces

1

0..*

1

0..*0..*

0..*

2..*

0..*

+Input

+Output

Refinement: Product

Product

Signal

Individual Object

Track

Situation Object

Object Act ion Effect

This is just a first refinement step, not even complete …

OODA

OODA in UML

Process

Decision0..*0..*

Orientation

0..*0..*

+basis

Observation

0..*0..*

+theObservation

+decisionFeedback

+guidanceAction

0..*0..*

+theAction

+actionFeedback

Refinement: Function(just an example)

Function

Output

getInput()

SubObject Assessment

Object Assessment

Situation Assessment

Impact Assessment

Orientation

• ontology 1. the branch of metaphysics dealing with the nature of being, reality or ultimate substance (cf. phenomenology) 2. particular theory about being or reality

• phenomenology 1. the philosophical study of phenomena, as distinguished from ontology 2. the branch of a science that classifies and describes its phenomena without any attempt at metaphysical explanation

• metaphysics 1. the branch of philosophy that deals with first principles and seeks to explain the nature of being or reality (ontology); it is closely associated with the study of nature of knowledge (epistemology)

Ontology (Webster)

• An explicit specification of a conceptualization: the objects, concepts, and other entities that are assumed to exist in some area of interest and the relationships that hold among them (Genesereth & Nilsson, 1997)

• Definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. A statement of a logical theory. (Gruber)

• An agent commits to an ontology if its observable actions are consistent with the definitions in the ontology (knowledge level).

• A common ontology defines the vocabulary with which queries and assertions are exchanged among agents.

Ontology (cont.)

Ontologies and IFRM

• IFRM is at the same level as UML and OWL (from the linguistic point of view)

• IFRM is a domain-specific (information fusion) modeling language– Ontology defines a conceptualization, a

vocabulary …

• Thus IFRM is an ontology, provided it is specified explicitly in a formal language (formal axioms)

Ontologies and Fusion System

MetaModel

Model

Objects Annotations

IFRM = Ontology of FusionIFRM

Model ofFusion System

Fusion System(run time)

Specific ontology

Commercialization Idea

• Achieve acceptance by the fusion community• Achieve acceptance by the Government• Request for Proposals (RFP) for standards -

the OMG• Work with OMG towards standard (IFRM)• Promote the standard• Build supporting tools• Result: Interoperability of various fusion

systems!

Tasks

Task 1 2 3 4 5 6 7 8 9

A. Analyze other models

B. Refine the model

C. Formalize top-level model

D. Develop evaluation methodology

E. Propose software tools

F. Identify evaluation scenarios

G. Prepare RFP to OMG

Backup Slides

Ontology Languages

• Web Ontology Language (OWL)– Lite– DL (Description Logics)– Full

• OWL+SWRL (Semantic Web Rule Language)

• RuleML

• First Order Predicate Calculus (FOPL)

• Higher Order Logics (HOL)

OWL & SWRL

XML

RDF

RDFS

OWL

SWRL

Extensible Markup Language

Resource Description Framework

RDF Schema

Web Ontology Language

Sem. Web Rule Language

Note: The layering of OWL on top of RDFS is not strict.

How many ontologies?

• Net Centric Warfare (NCW) requires full interoperability and thus communication

• An agent commits to an ontology if its observable actions are consistent with the definitions in the ontology (knowledge level).

• Two agents must commit to the same ontology in order to communicate

• What if there are N agents?– One common ontology? Seems impossible!– N2 ontologies? Unmanageable!– Ontology mapping? Work in progress.– Core ontology + extensions? Compromise.

• Somewhat similar idea to NATO’s Generic Hub (GH5) and C2 Information Exchange Data Model (C2IEDM)

C2IEDM Core

GH 5 Core

CANDIDATE-TARGET-LIST

CAPABILITY

RULE-OF-ENGAGEMENT

REPORTING-DATA

LOCATIONOBJECT-TYPE OBJECT-ITEM

CONTEXT

ACTION

GH5 Core

• A neat idea– Common core, extendable– Relatively rigorous– Substantial amount of knowledge– XML Schema available

• Not an ontology– Not formal (no formal semantics)– Thus cannot be processed by logical tools– Not quite compatible with MMF (mixes Object and

Model levels)– But looks like a good start towards an ontology

VIS SAW Core Ontology

SBIR Phase II, AFRL/IF Rome, Mike Hinman, John Salerno

Battlefield Ontology

What is “extension”?

• Add classes– E.g., Battalion is a Unit

• Add properties (relations)– E.g., Platoon is part-of Company

• Add constraints– E.g., Soldier can be part of only one

Platoon

• But the result must be an ontology that is consistent

Consistency

con•sis•ten•cy

agreement with what has already been done or expressed;conformity with previous practice

[Webster’s]

In logic: from P and not(P) can derive anything

Inconsistency is a dangerous thing for autonomous agents!

Inconsistency: Example

• Battlefield Ontology happens to be consistent• Suppose we have

– Constraint: Unit must have at least 8 Soldiers

• Suppose we then extend it by adding:– Class: Group (sub-class of Unit)– Constraint: Group has 3 Soldiers

• Inconsistency!• Easy to generate inconsistencies while developing

ontologies• ConsVISor to the rescue: http://www.vistology.com

Ontology Mapping

DB1 DBd KB1 KBkSsS1

Tt Oo

T1 O1

AD1 ADd AK1 AKk

ATt

AO1

AS1 ASs

AT1

AOo

GUI

Agents: commit to ontologies; negotiate mapping; use templates

Ontology Mapping (cont.)

• Need to map:– Classes to Classes– Properties to Properties– Objects to Objects– Constraint mapping implicit

• May result in inconsistency!

• ConsVISor to the rescue: http://www.vistology.com

Why Use Ontologies?1. Represent theories of potential objects and relations as ontologies (OWL)

2. Represent collected data as annotations in terms of ontology (OWL)

3. Formulate any queries about situations in OQL (OWL Query Language)

4. Use a general purpose OWL reasoner to answer queries

5. Use the trace of the reasoner to give an explanation to user

6. Multiple ontologies may need to be combined into one (fusion)

7. Data Association and Fusion

1. association of objects with ontologies/annotations

2. relations among objects within ontologies/annotations

3. combining ontologies/annotations using colimit of category theory

Flexibility: Can use the same reasoners on any ontologies and annotations!

Use Case: Ontology-Based Fusion

Use Case: Fusion System Development