CACI Private Data; Do Not Copy or Distribute Information Fusion Technical Area Overview &...
-
Upload
peregrine-horn -
Category
Documents
-
view
214 -
download
0
Transcript of CACI Private Data; Do Not Copy or Distribute Information Fusion Technical Area Overview &...
CACI Private Data; Do Not Copy or Distribute
Information Fusion Technical Area Overview & Applications
Joseph A Karakowski
(732)460-7752
November 16, 2011
CACI Private Data; Do Not Copy or Distribute
What we will cover..
the “important” parts…..about fusion
CACI Private Data; Do Not Copy or Distribute
Agenda
• Background Technology– Fusion Definition– Fusion Models
• Fusion Technology Sector Applications– Military– Medical & Non-Military
• Personal Fusion Areas (Optional)
Questions to be Answered:
What is Fusion Technology and it’s basis?
What are some example fusion applications in specific markets?
CACI Private Data; Do Not Copy or Distribute
Some Fusion Definitions……the process of combining data/information to
estimate or predict the state of some aspect of the world (Bowman)
…the process of utilising one or more data sources over time to assemble a representation of aspects of interest in an environment (Lambert)
…series of processes performed to transform observational data into more detailed and refined information, knowledge, and understanding (USArmy)
…everything is Connected… a “Global Graph” portrays the connected world; graph nodes are the entities; graph links are the actions or relationships (Walsh)
CACI Private Data; Do Not Copy or Distribute
What is Fusion?
Technical methods/processes which supports, through cognitive/perceptual modeling, the solution of a class
“Difficult Problems”
Some Typical characteristic of Difficult Problems:• Multiple Goals• Complexity, with large numbers of items, interrelations and decisions• Dynamic, time considerations• Cognitive/perceptual problem solving
These are a first scientific step to solve these classes of problems, which have not been solvable, up to this time.
Implementation of these processes using information technology, has been moving forward for the last 25+
years, and will probably continue for many more years…
CACI Private Data; Do Not Copy or Distribute
“The Blind Man & the Elephant”Question: What is an Elephant?
CACI Private Data; Do Not Copy or Distribute
It’s the JDL
Model !
It’s a Cognitive/Perceptual Process!
It’s Intelligence
Apps !
The “Fusion Elephant”
It’s a Global Graph !
Its Biometric
Apps!
A State Prediction Problem!
Question: What is Fusion?
CACI Private Data; Do Not Copy or Distribute
A More Realistic “Fusion Elephant”
Nuclear
This is a new technology, and
much RD&E remains
CACI Private Data; Do Not Copy or Distribute
Major Fusion Process Models
• Joint Directors of Laboratories Model (JDL)* [1986-Pres]
• Transformation of Requirements for Information Process (TRIP) Model [2000-?]
• Visual Situation Assessment Model () [1997]
• Salerno SA Model [2001-Pres]
• “Graph” Fusion Model [2005-Pres]
• Contextual Fusion Model * [2009-Pres]
• There are many others….
Many Definitions…and many more models have been proposed and built!
CACI Private Data; Do Not Copy or Distribute
ProcessRefinement
Level 4
SourceInput
Human/ComputerInteraction
Preprocessing/Predetection
FusionLevel 0
Single ObjectRefinement
Level 1
Location;Attributes
Behavior;Class; ID
Aggregateobject
refinement
Situationinterpretation
Intent;Vulnerability
Courses ofAction
SituationRefinement
Level 2
Implications/Threat
RefinementLevel 3
Database Services
Relatively statica priori
Knowledge
DynamicSituationDatabase
DFG Functional Model (JDL Model)
Richard Antony, in DFG Meeting Minutes, W. Doig, 14 March 1997
CACI Private Data; Do Not Copy or Distribute
Conceptually organized along three(related) dimensions: (entity, context1, context2)
AKA “Triple”
“Fusion” ..an “assessment” operation between pairs of Triples: Lead to 8 fundamental classes of fusion operations
Antony & Karakowski Contextual Fusion Model (CFM) 2009
CFM explicitly fuses diverse context with specific basic entities, all within a computational JDL model framework,
resulting in a testable, expandable, and general fusion model
“Fusion as a Process” exhausts all possible “assessment” combinations or fusion in a Triple; the result is a set of discovered concepts & relations from the fusion of the pairs of Triples. This provides a rich discovery space within an existing knowledge source
CACI Private Data; Do Not Copy or Distribute
12
• Information for fusion requires both context and entity• “Entity” is the specific unit of information, or node in a
graphical representation • Context allows perception of an Entity with respect to the
information of interest– Context gives meaning to an Entity’s “information”– Context is required before an information entity has any
meaning– Context must be an integral part of the fusion process
& process model, its computation paradigm
Context is knowledge that enhances the more complete understanding of a specific entity of interest and the desired resultant objective information product (s)
Contextual Fusion Model Context & Content
CACI Private Data; Do Not Copy or Distribute
Two Entities within a graph, with Two sets of Two Entities, as their Contexts
Green = Entity
E2
E3/C3 E4/C4
E1
E5/C5 E6/C6
Entity Graph Nodes
Graph Entity-Entiy Relations
Entities as Context
CACI Private Data; Do Not Copy or Distribute
14
FormIndividual
Entity Location TimeExample Concept of Fusion Result formed for Location and time context
1 Similar Similar SimilarFuse multiple, similar information sources;
tracking using E-sensors
2 Similar Similar Dissimilar Static similar object
3 Similar Dissimilar Similar Infeasible condition – inconsistency; a truth
maintenance signal
4 Similar Dissimilar DissimilarEntity tracking/ tracking using both (slower) E-
sensors and messages
5 Dissimilar Similar SimilarAssociation / correlation of possible action as co-
located Entities
6 Dissimilar Similar Dissimilar Association / correlation of entities based on
same location only
7 Dissimilar Dissimilar Similar With prior Communication info: Potential Entity
comm link (cell phone, chat, email)
8 Dissimilar Dissimilar Dissimilar
Multiple different entities at different times- further data mining and other FF conceptual analyses may be indicated
Fusion Operation = (entity A, location A , time A) x (entity B, location B, time B)
Fusion Operation = fusion of two entities with associated context
Level 1 Fusion
Level 2 Fusion
Eight Canonical Fusion FormsIntelligence Traditional Tracking/Correlation Application
Contextual Dimensions Similarity/Dissim Assessment Op
CACI Private Data; Do Not Copy or Distribute
Simple Physical Context Example - Voltage
Voltage
time.
Human Entity, for example, a much more complex entity…. This is like a generalization from “humans” to “human signals”
If a voltage(entity) is viewed by itself without any context, we just “see” a
value, either static, “semantic” or apparently varying
If voltage has added the context of time, “signals” are created, with the field of electrical electronic engineering and associated signal analysis….Note the huge information content difference between the entity of “voltage” and the addition of the context “time” and how context gives much more “meaning” to the entity (voltage)
CACI Private Data; Do Not Copy or Distribute
16
FormV-SourceCompare Location Time
Concept of Fusion Result of Voltage Sourcefor Location and time context
1 Similar Similar Similar Instantaneous single source value, at one place
2 Similar Similar Dissimilar Time varying single source value, at one place
3 Similar Dissimilar Similar Instantaneous V-field for single source
4 Similar Dissimilar DissimilarInstantaneous time-varying V-field for single
source
5 Dissimilar Similar SimilarInstantaneous multiple source value, at one place
6 Dissimilar Similar Dissimilar Time varying / multiple sources value, at one
place
7 Dissimilar Dissimilar Similar Instantaneous V-field for multiple sources
8 Dissimilar Dissimilar DissimilarInstantaneous time-varying V-field for multiple
sources
Fusion Operation = ( Source V1, location V1 , time V1) x (Source V2, location V2, time V2)
Level 1 Fusion
Level 2 Fusion
Eight Canonical Fusion FormsSource “Voltage” (just for fun)
Contextual Dimensions
CACI Private Data; Do Not Copy or Distribute
17
Individuals
Organizations
All Relations based on
Location & Time Context Only
Events
Prior Art: Military Target Entity Model
DRs of Individuals to EventsLevel 2
DRs of Individuals to Organizations
Level 2
DRs of Organizations
to EventsLevel 2
DRs of Events to
Events
DRs of Organization
to Organization
DRs of Individuals
to Individuals
DR: Discovered Relations thru contextual FFs
Other forms of discovery are possible;
(I O, OE ) (EI ) eg
CACI Private Data; Do Not Copy or Distribute
Fusion Applications
CACI Private Data; Do Not Copy or Distribute
Some Fusion Military Application Areas
• Intelligence
• Bio sensing/biometrics
• Situation Awareness
• Imagery
• SIGINT(COMINT/ELINT)
• Tracking
Can support at all levels: Hardware,
Software, Level 0 – Level 5
CACI Private Data; Do Not Copy or Distribute
Non-Military Fusion Application Areas
• Networking/Cellular
• Homeland Security
• Medicine
• Chemistry
• Cognitive sciences
• …many others…
CACI Private Data; Do Not Copy or Distribute
Fusion “Topics” from a recent conference…
CACI Private Data; Do Not Copy or Distribute
Conference [Shortened-”C”] Index of Fusion Topics I
• Camera• Capability Acquisition Graph• CBRN data fusion• Cellular automata• Centralized processing
systems• Challenge Problem Set• Change detection• Chemical plume• Classification fusion• Classification System• Closest point approach
• Clustering algorithm• Clutter• Co-ranking• Coalition formation• Coalition operations• Coarsening• Coastal radar• Cognitive Radio Networks• Collaborative systems• Collision mitigation• Color Clustering• ……
CACI Private Data; Do Not Copy or Distribute
• Combination of belief functions
• Combinatory categorial grammar
• Communication Decision• Communication failures• Complex object recognition• Compression• Computer security• Conceptual graphs
• Conditional independence• Confidence management• Configuration• Conflict analysis• Confusion• Conjunctive operator• Connection Model• Context• Contradiction• Convex optimization• Convoy tracking• Cooperative systems• Coordinate registration
Conference [Shortened-”C”] Index of Fusion Topics II
CACI Private Data; Do Not Copy or Distribute
• Coordination• Correlation• Course Of Action• Covariance control• Credal networks• Credibility• Crop modeling• Cross correlation• Cross-cueing• Cubic Spline Curve• Cued Sensors• Cyber fusion• Cyber-security
Conference [Shortened-”C”] Index of Fusion Topics III
From these three slides one can see both very specialized areas and much broader areas
which currently utilize information Fusion
technology
CACI Private Data; Do Not Copy or Distribute
Overview of Specific IF Apps from Selected market areas
Military1…Biometrics2…Target Detection & Tracking3…Chemical & Explosives4…Image Fusion
Medical5…Breast Cancer6…Radiology
Non-Military 7…Dept of Homeland Security8…Cyber Security
Summaries of specific fusion papers follows…
Note: All these apps will fall somewhere in the fusion models and
fusion definitions which I previously
described.
CACI Private Data; Do Not Copy or Distribute
1. BiometricsA Multibiometric Face Recognition Fusion
Framework with Template Protection[1]• “A fusion framework.. which demonstrates how …algorithms that
produce hard decisions can be combined with unprotected algorithms that produce scores or soft decisions”
Military & Commercial
Improving the recognition of fingerprint biometric system using enhanced image fusion[2]•“approach to increase the verification and identification of fingerprint recognition. This was achieved by using … linear fusion techniques”
Multimodal Eye Recognition[3]• “results show that the proposed eye recognition method can achieve better performance…, and the accuracy of…kernel-based matching score fusion methods is higher than PCA and LDA”
CACI Private Data; Do Not Copy or Distribute
2. Target Detection &Tracking
Level 0-2 fusion model for ATR using fuzzy logic[4]
• “use of fusion at the lowest levels has been demonstrated. …provides a structure for fusion of multispectral data at all levels”
Military
Long-duration Fused Feature Learning Aided Tracking[5]• “Our experiments indicate that the Long-term Hypothesis Tree algorithm, which solves the tracklet-to-tracklet association problem, can be used to strongly disambiguate a multitude of situations and is a more computationally efficient algorithm than previously proposed joint solutions”
CACI Private Data; Do Not Copy or Distribute
3. Chemical & Explosives
Fusing chlorophyll fluorescence and plant canopy reflectance to detect TNT contamination in soils[6]
• “physiological response of plants grown in TNT contaminated soils and … to detect uptake in plant leaves…use remote sensing of plant canopies to detect TNT soil contamination prior to visible signs”
Military Market
Sensor data fusion for spectroscopy-based detection of explosives[7]• “Multi-spot fusion is performed on a set of independent samples from the same region…. Furthermore, the results … are fused using linear discriminators. Improved detection performance with significantly reduced false alarm rates is reported using fusion techniques”
CACI Private Data; Do Not Copy or Distribute
4. Image Fusion
Towards Visual-Data Fusion[8]• “Fusion for both data and visual processes are derived as
specific transforms from human linguistic requests. Visual “understanding” occurs by human-directed perception of summarized pattern representations within a familiar frame of reference”
Military & Commercial
An orientation-based fusion algorithm for multisensor image fusion[9]• “Gabor wavelet transform … to fuse visible images and thermal images; orientation-based fusion is superior to the results of multiscale fusion algorithms…and can be applied to multiple (more than two) image fusion”
CACI Private Data; Do Not Copy or Distribute
5. Breast Cancer
Investigation of PET/MRI Image Fusion Schemes for Enhanced Breast Cancer Diagnosis[10]
• “results indicate that the radiologists were better able to perform a series of tasks when reading the fused PET/MRI data sets using color tables generated by our new genetic algorithm, as compared to commonly used …schemes”
Medical
Time of Arrival Data Fusion Method for Two- Dimensional Ultrawideband Breast Cancer Detection[11]•“A new microwave imaging method is given for breast tumor detection using an ultrawideband (UWB) imaging system. By combining the time of arrival (TOA) measurements from different sensors, the presence and location of small malignant lesions can be identified”
CACI Private Data; Do Not Copy or Distribute
6. Radiology
KNOWLEDGE BASED FUZZY INFORMATION FUSION APPLIED TO CLASSIFICATION OF ABNORMAL BRAIN TISSUES FROM MRI[12]
• “automatically classify abnormal tissues in human brain in a three dimension space from multispectral magnetic resonance images such as TI-weighted. T2- weighted and proton density feature images. It consists of four steps: data matching. information modeling, information fusion and fuzzy classification”
Medical – Add
New Applications of Planar Image Fusion in Clinical Nuclear Medicine and Radiology[13]• Fusion of multiple modalities has become an integral part of modern imaging methodology, especially in nuclear medicine where PET and SPECT scanning are frequently paired with computed tomography(CT). Additional fusing of orthopedic radiographs with photographic images of the extremities..
CACI Private Data; Do Not Copy or Distribute
7. DHSInformation Fusion for CB Defense Applications[14]• “With appropriate algorithmic approaches and appropriately resolved
tradeoffs, information fusion can offer… the potential of reaching performance that would be difficult, if not impossible, to attain otherwise. Thus, information fusion represents a significant
opportunity for the CB defense and homeland security realm”
Military & Non-Military
Decision-level Information Fusion to Assess Threat Likelihood in Shipped Containers[15]•“details an approach to the decision-level fusion of disparate information to produce an assessment of the presence of a threat in a shipping container”
Homeland Security Fusion Application of STEF[16] •“fusion system provided sufficient actionable intelligence that could have stopped a .. realistically staged terrorist attack on a US civilian target. …provided sufficient information to allow .. arresting the mastermind of the plot, as well as other key individuals and detaining the lower level individuals in his network, including the suicide bomber”
CACI Private Data; Do Not Copy or Distribute
8. Cyber Security
Application of the JDL Data Fusion Process Model for Cyber Security[17]
• “explores the underlying processes identified in the Joint Directors of Laboratories (JDL) data fusion process model and further describes them in a cyber security context”
Military & Non-Military
CACI Private Data; Do Not Copy or Distribute
We have covered “the more important parts”…a warning
CACI Private Data; Do Not Copy or Distribute
Summary & Closing Comments
• Short background of Fusion Technology & Models/Contextual Fusion Model
• Few examples of Fusion R&D / Apps
• A lot was left out ….
CACI Private Data; Do Not Copy or Distribute
Backups
CACI Private Data; Do Not Copy or Distribute
Some of My Personal Fusion RDE Areas
• UGS tracking/ID L0/L1• RADAR ID
– Signal processing L0/L1 / Fuzzy Expert– Confirmation/Disconfirmation
• Voice Fingerprint ID biometrics L0/L1• Visual fusion L0-L2[*]• Semantic/contextual unstructured information --
understanding & discovery L1-L3[*]• Contextual Fusion System[2006-2010]• General Context Fusion Model [2011-?]
CACI Private Data; Do Not Copy or Distribute
Some Fusion Publications
• Karakowski, J.A., “An Application of Text-Independent Speaker Recognition to High Speed Voice Surveillance”, Wide Area Surveillance Symposium, Office of Nat’l Drug Control Policy/Counter Drug Technology Assessment Center(1993)
• Karakowski, J.A., “Text Independent Speaker Recognition using A Fuzzy Hypercube Classifier”, ICASSP97(1997)
• Karakowski, J.A., “Towards Visual Fusion”, Invited Paper, Georgia Tech(1998).• Antony, R. T. and Karakowski, J. A., “Service-Based Extensions to the JDL Fusion
Model,” SPIE Defense Security and Sensing Conference (March 2008).• Antony, R. T. and Karakowski, J. A., “Fusion of HUMINT & Conventional Multi-Source
Data,” National Symposium on Sensor and Data Fusion, Session SC04 pp. 1-16 (07).• Antony, R. T. & Karakowski, J. A., (2007) “Towards Greater Consciousness in Data
Fusion Systems,” MSS National Symposium on Sensor and Data Fusion, (June 07).• Antony, R. T. and Karakowski, J. A., “First-Principle Approach to Functionally
Decomposing the JDL Fusion Model: Emphasis on Soft Target Data,” Fusion (July 08).• Antony, R. T. & Karakowski, J. A., “Discovery Tools for Soft Target Applications,”
National Symposium on Sensor and Data Fusion(2009)• Antony, R. T. and Karakowski, J. A., “First-Principles Mapping of Fusion Applications
into the JDL Model,” SPIE Defense Security and Sensing Conference (April 2009)• Antony, R.T, & Karakowski, J.A., “Multiple Level-of-Abstraction Tracking and Alias
Resolution”, National Symposium on Sensor and Data Fusion(2010)• Antony, R.T., & Karakowski, J.A., “Toward more Robust Exploitation of the Asymmetric
Threat: Binary Fusion Class Extensions”, (April 2011) SPIE.