Visualization, Level 2 Fusion, and Homeland Defense Dr. James Llinas Research Professor, Director...
-
Upload
manuel-falkner -
Category
Documents
-
view
215 -
download
1
Transcript of Visualization, Level 2 Fusion, and Homeland Defense Dr. James Llinas Research Professor, Director...
Visualization, Level 2 Fusion, Visualization, Level 2 Fusion, and Homeland Defenseand Homeland Defense
Dr. James LlinasResearch Professor, Director
Center for Multisource Information Fusion
University at Buffalo
OutlineOutline• Overview of a DARPA-sponsored Workshop
on :– “Ontology Definition and Development, and the
Perceptual/Comprehension Interface for Military Concepts”
• Remarks on Visualization Challenges of Homeland Defense
The WorkshopThe Workshop
--Ontology Action Plan
--Perspectives on Visualization (Kesavadas)
Workshop AssertionWorkshop Assertion• The Data Fusion community is progressing toward
meaningful achievements in Level 2 and 3 fusion processing capability—but there is no community ontology for the L2/L3 products*--a process must be started to assess the need for, nature of, and means to achieve a supporting, consensus L2/L3 Ontology (or Ontologies) that yields the important benefits associated with ontologically-grounded systems, such as Interoperability, Semantic Consistency, Completeness, Correctness, Adaptability, etc
* To include “Threat States”, “Intent”, etc.
Data Fusion Functional Model(Jt. Directors of Laboratories (JDL), 1993)
Level 0 — Sub-Object Data Association & Estimation: pixel/signal level data association and characterization
Level 1 — Object Refinement: observation-to-track association, continuous state estimation (e.g. kinematics) and discrete state estimation (e.g. target type and ID) and prediction
Level 2 — Situation Refinement: object clustering and relational analysis, to include force structure and cross force relations, communications, physical context, etc.
Level 3 — Impact Assessment: [Threat Refinement]: threat intent estimation, [event prediction], consequence prediction, susceptibility and vulnerability assessment
Level 4: Process Refinement: adaptive search and processing (an element of resource management)
Level 0Processing
Sub-object DataAssociation &
Estimation
Level 1Processing
Single-ObjectEstimation
Level 2Processing
SituationAssessment
Level 3Processing
Threat/ImpactAssessment
Level 4ProcessingAdaptive Process
Refinement
Data BaseManagement System
SupportDatabase
FusionDatabase
INFORMATION FUSION PROTOTYPEJEM
JWARN3GCCS
JWARN3
GCCS
JEM• CBRN Point and
Standoff Sensors• CBRN Data
Sources• Intel Sources• Air Surveillance• Surface Sensors• Standoff Sensors• Space
Surveillance
• Reliability• Improved Detection
• Extended Coverage(spatial and temporal)
• Improved SpatialResolution
• Robustness (Weather/visibility, Countermeasures)
• Improved Detection
• Improved State Estimation (Type, Location, Activity)
• DiverseSensors
• MultiplePlatform Sensors
• MultipleSensors
Operational Benefits of Multiple SensorOperational Benefits of Multiple SensorData FusionData Fusion
State Estimates of Reduced UncertaintyAnd Improved Accuracy
DetectionTracking
ID
AggregationBehavior
Events
LethalityIntent
Opportunity
Sensor MgmtProcess Mgmt
Methods:--Combinatorial Optimization
--Linear/NL Estimation--Statistical
--Knowledge-based--Control Theoretic
Ontology-Based Fusion & Visualization*
“Raw Data”(Truly raw and also L1 estimates)
Associated to Ontologically-Based L2 Fusion Process
The Results of WhichProvide the Raw Material
For Visualization
* Ontology-based Information Visualization, F. vonHarmelen, et al, Proceedings of the workshop on Visualization of the Semantic Web (VSW'01)", 2001
(Which we don’t have)
Visualization Challenges:--the Ontology itself (presuming it is large and complex)
--the L2 fusion results (complex, high-dimensional, abstract concepts, not spatially referenced)
An Ontology Action Plan for An Ontology Action Plan for the Information Fusion the Information Fusion
Community:Community:Results of a DARPA/CMIF Workshop, Nov. 2002
Dr. James Llinas
Dr. Eric Little
Center for Multisource Information Fusion
University at Buffalo
BackgroundBackground
• Analysis and Decision-Support Needs for New and Diverse defense and national-security problems are demanding major improvements in Level 2 and 3 Information Fusion (IF) capabilities.
• U.S. and International efforts are underway to address many of the foundational issues associated with achieving such IF capability, especially system architecture and algorithmic processing.
• However, the topic of Ontological Requirements as a foundation for these L2, L3 initiatives has not been explicitly addressed, although it is agreed that many Ontologically-related activities are underway to include Ontological prototyping but largely addressed from a Computational Ontology point of view.
• In addition, the abstract nature of many L2, L3 information products also places a demand on the approach to and means for Visualization of such fusion products.
• In November 2002, a Workshop sponsored by DARPA and the CMIF was held to address these latter two issues.
• This briefing summarizes thoughts from the Workshop regarding the Ontology topic only.
Ontology TrackOntology Track
A Tentative ConclusionA Tentative Conclusion
• This Workshop opened with the following assertion:
• This assertion, and the higher-level, implied assertion that “Good Ontologies Yield Good Fusion Systems”, was conditionally accepted by the Workshop attendees.
• The conditional aspects revolved about the need for some type of experimental proof—there was a consensus on the need for:– A Proof-of-Concept Demonstration / Experiment
– Definition and Employment of Appropriate Metrics and Evaluation Procedures that Quantify:
• Ontology Quality Per Se
• “Good” Ontology’s Contribution to Superior Fusion System Performance
• These activities would comprise just a part of a larger Action Plan.
• The Data Fusion community is progressing toward meaningful achievements in Level 2 and 3 fusion processing capability—a process must be started to assess the need for, nature of, and means to achieve a supporting, consensus Ontology (or Ontologies) that yields the important benefits associated with ontologically-grounded systems, such as Interoperability, Semantic Consistency, Completeness, Correctness, Adaptability, etc
Ontology-Related Track: Ontology-Related Track: Key Issues for an Action PlanKey Issues for an Action Plan
• An Action Plan for Ontology—What have we learned?– Do we agree there is a need for a consensus ontology?
– Gauging the nature and size of the underlying Taxonomy:• The issue of “Admission” to the Taxonomy
• The issue of the Extent of the Taxonomy
– Formal Ontological Methods:• Degree of formalism required
– Accommodating a Hybrid approach
– Research issues
– Consensus-forming• Approach
• “Configuration Control”, once a baseline is established
– Construction Methods• General approach
• Automated Tools
Nature and Size of the L2, L3 TaxonomyNature and Size of the L2, L3 Taxonomy
• Nature: “Admission” to the Taxonomy– Coarse Filter: In the main, L2 is about Situational Assessment,
and L3 is about Threat and Impact Assessment, and we can easily populate that portion of the taxonomy
– Fine Filter: To be determined• Candidate Approach: Build on the OSD/Decision Support Center’s
study of Essential Elements of Information (EEI’s)– Cost-Efficient
– EEI’s well received by operational community
– Conduct initial analysis before next workshop
• Incorporate pre-workshop taxonomy
• Size: estimated as a subset of 3700-long EEI list, TBD
Formality in Ontology-DevelopmentFormality in Ontology-Development
• Methods for formal ontology development exist—but--• Degree of formality fundamentally depends on Ontology
Requirements– Develop from a Systems-Approach– Need to build both application-requirements and technical
requirements• Application: Requires defining Role for Ontology in IF applications
– Human understanding– Computational benefits– Performance/Effectiveness benefits
• Technical: Requires quantifying technical criteria of goodness:– Consistency– Completeness– Accuracy– etc
Selecting the Level of Selecting the Level of FormalityFormality
Ca
talo
g
Te
rms/g
lossa
ry
Th
esa
uri
(“n
arr
ow
er
term
” re
latio
n)
Info
rma
l “i
s-a”
hie
rarc
hy
Fo
rma
l “i
s-a”
hie
rarc
hy
Fo
rma
l in
sta
nce
Fra
me
s (
pro
pe
rtie
s)
Va
lue
re
str
ictio
ns
Dis
join
tne
ss, in
ve
rse
, e
qu
ivale
nce
, e
tc.
Ge
ne
ral lo
gic
al co
nstr
ain
ts
Ontologies
De
we
y D
ecim
al
Sys
tem
Da
ta d
ictio
na
ry
Ya
ho
o d
ire
cto
ry
Bio
logic
al ta
xo
no
my,
RD
B s
ch
em
a,
OO
DB
sch
em
a
XM
L s
ch
em
a
CL
OS
, C
++
, Ja
va
, R
DF
UM
L
DA
ML
+O
IL
(OW
L)
DA
ML-L
(ru
les),
K
IF
Te
xt
se
arc
h
en
gin
e t
he
sa
uri
Taxonomies
Ex
pre
ss
ive
ne
ss
Ca
talo
g
Te
rms/g
lossa
ry
Th
esa
uri
(“n
arr
ow
er
term
” re
latio
n)
Info
rma
l “i
s-a”
hie
rarc
hy
Fo
rma
l “i
s-a”
hie
rarc
hy
Fo
rma
l in
sta
nce
Fra
me
s (
pro
pe
rtie
s)
Va
lue
re
str
ictio
ns
Dis
join
tne
ss, in
ve
rse
, e
qu
ivale
nce
, e
tc.
Ge
ne
ral lo
gic
al co
nstr
ain
ts
Ontologies
De
we
y D
ecim
al
Sys
tem
Da
ta d
ictio
na
ry
Ya
ho
o d
ire
cto
ry
Bio
logic
al ta
xo
no
my,
RD
B s
ch
em
a,
OO
DB
sch
em
a
XM
L s
ch
em
a
CL
OS
, C
++
, Ja
va
, R
DF
UM
L
DA
ML
+O
IL
(OW
L)
DA
ML-L
(ru
les),
K
IF
Te
xt
se
arc
h
en
gin
e t
he
sa
uri
Taxonomies
Ex
pre
ss
ive
ne
ss
Integrated Data FusionDictionary for the designers, users
Computational Ontology suitable for automated
reasoning
Ontology suitable for structured data
managementfrom: Deborah McGuiness, “Ontologies Come of Age”
Consensus-FormingConsensus-Forming• Approach Options Nominated :
– NATO STANAG-development process– Via Int’l Society for Information Fusion (ISIF)– U.S. DoD lead but International in scope
• Link to Computer Science community via:– Open Source Consortium– IEEE, ACM
• Link to Int’l Community Required: eg, Canadian and Australian IF communities are addressing Ontological matters; TTCP and NATO both active
• Broad communication, coordination required:– Website(s)– VTC’s– Use of CSCW technology– Specialized Conference sessions
• Once Requirements have been specified, those reqmts either directly or indirectly influence the overall approach to Ontology construction, eg:
– Formalism
– Language
– Automated Tools
– Tools for Visualizing the Ontology
– Strategies for Ontology evaluation
• In the following we borrow directly from the paper by Anne-Clair Boury-Bisset and M. Gauvin: OntoCINC Server: A Web-based Environment for Collaborative Construction of Ontologies, 19 Sept 2002*
Ontology ConstructionOntology Construction
•Anne-Claire was a workshop attendee and briefed the attendees on the cited topic
Ontology Construction ApproachOntology Construction Approach
1. Identification of the task for which the ontology is being developed;
2. Definition of the requirements for the ontology: purpose and scope;
3. Informal specification: Build informal specification of concepts;
3. Encoding: Formally represent the concepts and axioms in a language;
5. Evaluation of the ontology.
1.ID Data Fusion Ontology Task – ID Military Utility
2.Data Fusion Ontology Purpose, Scope, Formality
3.Build Taxonomy; then specify concepts
4.CollaborativeDevelopment
SelectTool
5. Evaluate DF Ontology
RealWorld
Validate
VerifyUtility
Ontology ConstructionOntology Construction• From Boury-Bisset, Gauvin:
Defence R&D Canada – Valcartier • R & D pour la défense Canada – Valcartier
Ontology development methodologies
• Most important methodologies– The TOVE (Toronto Virtual Enterprise) project [Uschold
andGruninger, 96]
– The Enterprise ontology [Uschold and Gruninger, 96]
– The Methontology method [Fernandez et al, 97]
– The IDEF5 ontology capture method [KBSI, 94]
• Main stages– Identification of the task for which the ontology is being developed;
– Definition of the requirements for the ontology: purpose and scope;
– Informal specification: Build informal specification of concepts;
– Encoding: Formally represent the concepts and axioms in a language;
– Evaluation of the ontology.
• From Boury-Bisset, Gauvin:
Defence R&D Canada – Valcartier • R & D pour la défense Canada – Valcartier
Comparison of ontological engineering tools
Evaluation criteria / tools
OntolinguaServer
WebOnto Protégé 2000 OntoSaurus ODE
High-level primitives
Primitives from the Frame Ontology
Language OCML. Many primitives available
OKBC knowledge model
LOOM language Conceptual level: concepts, relations, axioms, subclass-of, etc.
Existence of Libraries
Important repository of ontologies
Many examples Not available Yes Examples only
Interface Good graphical interfaceDifficult to use
Very goodClear
Good interface. Customizable layout
Ontology browsing mainly.
Not available for evaluation
collaborative ontology buiding
Best tool. Users, groups.R/W accessNotification
Synchronous cooperation: lock function.Broadcast / receive
Not a collaborative tool
Lock function for collaborative editing
Asynchronous cooperation only
Config. Web Server Web Server Local installation Both Local installation
Import / Export
Many formalisms: CLIPS, LOOM, CORBA’s IDL
No export JDBC DatabaseRDF
Export in Ontolingua, KIF, IDL.
Export: Ontolingua, F-Logic
Ontology ConstructionOntology Construction
• From Boury-Bisset, Gauvin:
Ontology ConstructionOntology Construction
• Web-based collaborative environment
• Flexible Meta-model
• Dynamic configuration of the environment
• Knowledge-level modelling
• Ontology editing and discussing
Requirements for an Ontology-Development Tool
Defence R&D Canada – Valcartier • R & D pour la défense Canada – Valcartier
Collaborative Ontology Development using OntoCINC Server
TOOL
FORMALISM
ROLE
METHOD
Ontology DevelopmentEnvironment
Decisions
Comments Forum discussion
Concepts
Attributes
Relations
Semanticmeaning
Synonyms
Metamodel management
Presentation
Captureexpertise
Viewing Ontologies*
* http://gollem.swi.psy.uva.nl/workshops/ka2-99/camready/shum.pdf
Ontology Visualization*
* Ontology-based Information Visualization, F. vonHarmelen, et al, Proceedings of the workshop on Visualization of the Semantic Web (VSW'01)", 2001
Visualization of the Ontology*:Visualization of the Ontology*:A Consensus Development-Tool NeedA Consensus Development-Tool Need
12
Information ModelingInformation ModelingInformation Modeling
Note that only the types of possible relationships are predefined; not the contents of the objects themselves.
This makes possible the integration and concurrent analysis of an unlimited variety of information types
• Hybrid model captures multiple types of relationships that may exist among arbitrary “information objects”
• Supports both information retrieval and comparison operations
14
Starlight Global VisualizationsSStarlight Global Visualizationstarlight Global Visualizations
NetworkRelationships
Hierarchical Relationships
Text Similarity Discrete PropertyCorrelations
17
Concurrent Information AnalysisConcurrent Information AnalysisConcurrent Information AnalysisOntologies Inherently
Reflect Complex Interrelationships
Visualization of the Ontology Structure is Needed as a Construction Aid
Visualization Tools are Needed That can Show
Many, Complex Interrelationships
* J. Risch of Pacific-NW Battelle was also a workshop attendee and discussed Starlight’s capabilities; it is a capability reflective of the state-of-the-art in advanced visualization tools
Concurrent Information AnalysisConcurrent Information Analysisin Starlightin Starlight
SummarySummaryAction Plan TasksAction Plan Tasks
• Define Participants• Begin the Systems Engineering process for Ontology Development
– Task(s) within an Future Combat System scenario• Coordination with CECOM, DARPA
– Role• Coordination with CECOM, DARPA
– Ontology Requirements to include Formality requirements• Define also Visualization-Support Requirements and Visualization Interface
– Encoding– Test and Evaluation
• Reviewing “master” EEI-set as a foundation for an initial Taxonomy for L2, L3– Determine “coarse” and “fine” filters for EEI selection
• Defining and executing the proof of concept demo– Scenario: One of the approved FCS scenarios– Metrics and evaluation approach: TBD– Scope: TBD
• Develop an approach to Consensus-forming– Coordination with US, NATO, TTCP, ISIF
Visualization Challenges of Visualization Challenges of Homeland DefenseHomeland Defense
Homeland defense is protecting a nation-state’s territory, population and critical infrastructure at home by:
• Deterring and defending against foreign and domestic threats.
• Supporting civil authorities for crisis and consequence management.
• Intelligent Response and RecoveryIntelligent Response and Recovery
• Helping to ensure the availability, integrity, survivability, and adequacy of critical national assets.
• Planning and MitigationPlanning and Mitigation
US Army TRADOC White Paper: http://www.fas.org/spp/starwars/program/homeland/final-white-paper.htm
Homeland Defense and WMD (CBRN)
• What’s different about WMD*?
– Situations not easily recognizable– Situations may comprise multiple, phased events– Most likely a complex (3D) urban landscape environment– Broad repertoire of input sources
• Typical: Multi-sensor/multi-source• Atypical: eg Epi-Intel (human, epizootic, food surety)
– Responders at high risk; that risk must be factored into response plan– Location of incident is a crime scene requiring evidence preservation– Subtle contamination-propagation must be accounted for– Incident scope may grow exponentially, stressing multi-jurisdictional
resources– Strong public reaction; fear, panic, chaos, anger– Time critical– Responder facilities may in fact be targets; eg PSAP’s
* United States Government Interagency Domestic Terrorism Concept of Operations Plan
Homeland Defense Applications
Visualization Examples: WMD and InfoWar
Urban LandscapesTerraSim - Philly & Pittsburgh examples
Harris RealSite
Geometry is Auto-Extracted from Imagery
Building Sides textures generated from satellite, aerial, or handheld photos
Models can be hand enhanced as needed after auto-extraction
Interactive environmentInReality provides mensurationtests for information discovery from datasets
Urban Landscapes
Harris InReality
Mensuration - determines distance and line of sight from any point on the database
Urban Landscapes
3D CFD Chemical Plume DispersionCT-Analyst @ NRL
Chemical Agent DispersionSoftware Solutions and Environmental Services Company
Building Internal StructuresArmy Corps of Engrs
Subway ApplicationsArgonne Natl Lab + Sandia
Network Intrusion Detection
• “Ontology and Visualization of Data Fusion Concepts: Support to Command and Control in a Network-Centric Warfare Environment “
• Four Tracks:– Evaluation– Impacts of the Distributed Environment– Notion of Contextual Understanding– Homeland Defense Applications
• Dates: TBD, Summer or early Fall 2003• Location: Beaver Hollow Conference Center,
Java, NY
Next CMIF Workshop:Army-Sponsored
Ordnance Explosive Power from Remote SensingOak Ridge Natl Lab