L ESSON I: I NTRODUCTION David L. Hall. L ESSON O BJECTIVES Introduce this course and instructor ...
-
Upload
leonard-lawson -
Category
Documents
-
view
218 -
download
1
Transcript of L ESSON I: I NTRODUCTION David L. Hall. L ESSON O BJECTIVES Introduce this course and instructor ...
LESSON I: INTRODUCTION
David L. Hall
LESSON OBJECTIVES
Introduce this course and instructor Provide an understanding of the course
logistics, requirements, grading, assignments, and ground rules
Introduce the topic of data and information fusion
COURSE OBJECTIVES
To provide an introduction to the field of data and information fusion Models of multisensor data fusion The JDL Data Fusion Process Model Techniques for data fusion ranging from estimation to pattern
recognition and automated reasoning Guide you through a team exercise involving design of a data
fusion system to address a selected application Present a balanced view of the advantages and limitations of
fusion Understand the role of the human in the loop analyst/decision
maker Provide a basis for further study and specialization
REVEALING MY PEDAGOGICAL HAND
Brief presentations In-class (on-line) exercises Humor and stories On-line discussion &
presentations Planned time for group meetings
& work
It is important that you:1) Participate every week2) Focus on and complete on-line materials, lecture material, readings, assignments.
It is important that you:1) Participate every week2) Focus on and complete on-line materials, lecture material, readings, assignments.
ON-LINE MATERIALS
On-line Lessons
• Site navigation on LHS
• Each lesson summarizes• Video pre-view • Introduction • Lesson objectives • Commentary &
discussion• Activities &
assignments
• Links to readings (via electronic library reserve)
• Electronic copy of Lecture
materials
TEXT AND READINGS
Assigned Text
• D. Hall and S. A. McMullen, Mathematical Techniques in Multisensor Data Fusion, Artech House, 2004
Selected Readings
• D. Hall and J. Llinas, editors, Handbook of Multisensor Data Fusion, CRC Press, 2001
• Excerpts from selected textbooks• Selected technical papers• One science fiction story
COURSE LESSONS
1. Introduction2. JDL Model3. Project initiation4. Sensor processing5. Level 0 processing6. Level 1 – Correlation7. Level 1 – Estimation8. Level 1 – Target ID9. Systems Engineering
10. 10. Project design11. Level 2 (situation refinement)12. Level 3 (Consequence refinement)13. Level 4 – Process refinement14. Level 5 – Cognitive refinement15. Project detailed design16. Data Fusion state of the art17. Final Presentation
ASSIGNMENTS & WEIGHTS
Individual Assignments and weights (70 %)
• Ten (10) low stakes quizzes (20 %)• Six (6) on-line discussion participation (15 %)• Eight (8) Individual writing assignments (24 %) • Peer evaluation (11 %)
Group project and weights (30 %)
• Final technical report (20 %)• Final presentation (10 %)
GROUND RULES
Attendance/participation & preparation Plagiarism (cheating) Academic integrity Affirmative Action & Sexual Harassment Americans with Disabilities Act
You have one week to question or dispute grades, missed assignments, or missed classes:
Note – I dislike “wheedling” for extra credit
THE ORIGIN OF MULTISENSOR DATA FUSION
“I say fifty, maybe a hundred horses . . . What do you say, Red Eagle?”
BIOLOGICAL ORIGINS OF SENSOR FUSION
Sight
Smell
SonarTouch Chemical detection
Sound
AUGMENTATION OF SINGLE SENSES
A long history of single sense augmentation has included eyeglasses, hearing aids, telescopes, microscopes (and more recently electronic noses, chemical detectors and many others
AUGMENTATION OF COGNITION
Similar to the augmentation of our senses, a long history of effort has sought to augment out cognition
Data fusion seeks to support the augmentation & automation of the multi-sensing, cognition process for improved awareness and understanding of the world
JDL DATA FUSION MODEL
• Course organized using the JDL Model “Levels”
• Hall and McMullen text organized around JDL model
• Lessons in on-line site focus on JDL levels
• Lessons systematically “walk through” the levels
• Project focus on designing a data fusion system for selected application using the JDL framework
DATA FUSION FUNCTIONAL MODEL
The JDL model (1987-91) and the draft revised model (1997)
• Level 0 — Sub-Object Data Association and Estimation: pixel/signal level data association and characterization
• Level 1 — Object Refinement: observation-to-track association, continuous state estimation (e.g. kinematics) and discrete state estimation (e.g. target type and ID) and prediction
• Level 2 — Situation Refinement: object clustering and relational analysis, to include force structure and cross force relations, communications, physical context, etc.
• Level 3 — Significance Estimation [Threat Refinement]: threat intent estimation, [event prediction], consequence prediction, susceptibility and vulnerability assessment
• Level 4 — Process Refinement: adaptive search and processing (an element of resource management)
Adapted from A. Steinberg
JOINT DIRECTORS OF LABORATORIES DATA FUSION
SUBPANEL:
DEFINITION OF DATA FUSION:DEFINITION OF DATA FUSION:
A continuous process dealing with the association, correlation, and combination of data and information from multiple sources to achieve refined entity position and identity estimates, and complete and timely assessments of resulting situations and threats, and their significance.
DEFINITIONS . . .
• Sensor Fusion = Data Fusion from Multiple Sensors (same or different sensor types)
• Data Fusion = Combining information to estimate or predict the state of some aspect of the world
• Data Fusion Functions:– Data Alignment
(spatio-temporal, data normalization, evidence conditioning)
– Data Association (hypothesize entities)
– State Estimation & Prediction
(etc.)
Platform
(etc.)
Reports
Situation
Cross-Force Relations
Force Structure
Unit
Traditional Focus
REPRESENTATIVE DATA FUSION APPLICATIONS FOR DEFENSE
SYSTEMSSPECIFIC
APPLICATIONSINFERENCES
BY DF PROCESSPRIMARY
OBSERVABLE DATASURVEILLANCE
VOLUMESENSOR
PLATFORMSOcean Surveillance Detection, Tracking,
Identification ofTargets/Events
EM Signals Acoustic Signals Nuclear Related Derived Observations
(wake)
Hundreds ofNautical Miles
Air/Surface/Sub-Surface
Ships Aircraft Submarines Ground-based Ocean-based
Air-to-Air andSurface-to-AirDefense
Detection, TrackingIdentification ofAircraft
EM Radiation Hundreds of Miles(Strategic)
Miles (Tactical)
Ground-based Aircraft
BattlefieldIntelligence,Surveillance andTarget Acquisition
Detection andIdentification ofPotential GroundTargets
EM Radiation Tens to Hundredsof Miles about aBattlefield
Ground-based Aircraft
Strategic Warningand Defense
Detection ofIndications ofImpending StrategicActions
Detection/Trackingof Ballistic Missilesand Warheads
EM Radiation Nuclear Related
Global Satellites Aircraft
REPRESENTATIVE DATA FUSION APPLICATIONS (NON-DOD
SYSTEMS)SPECIFIC
APPLICATIONSINFERENCES
BY DF PROCESSPRIMARY
OBSERVABLE DATASURVEILLANCE
VOLUMESENSOR
PLATFORMSCondition-basedMaintenance
Detection, characterizationof system faults
Recommendations formaintenance/correctiveactions
EM Signals Acoustic Signals Magnetic Temperature X-rays
Microscopicinspection tohundreds of feet
Ships Aircraft Ground-
based (e.g.,factory)
Robotics Object location,recognition
Guide the locomotion ofrobot hands, feet, etc.
TV Acoustic Signals EM Signals X-rays
Microscopic totens of feetabout the robot
Robot Body
Medical Diagnosis Location, identification oftumors. abnormalities, anddisease
X-rays NMR Temperature IR Visual Inspection Chemical/Biological
Data
Human bodyvolume
Laboratory
EnvironmentalMonitoring
Identification, location ofnatural phenomena(earthquakes, weather)
SAR Seismic EM Radiation Core Samples Chemical/Biological
Data
Hundreds ofmiles
miles (sitemonitoring)
Satellites Aircraft Ground-
based Underground
Samples
WHAT DOES DATA FUSION DO?
OPERATORS &SUPPORT SYSTEMS
SENSORS
SOURCESMISSION
EQUIPMENT &WEAPONRY
MISSIONENVIRONMENT
OPERATES ON:• sensor data• processed data• reference data
DATA FUSION FUNCTIONS
• ASSOCIATION• ESTIMATION• PREDICTION• INFERENCING• ANALYSIS• ASSESSMENT
• positional, identity, and attribute estimates about objects and events
• situation refinement• refinement of enemy threats,
vulnerabilities, opportunities
TO SUPPORT
HIERARCHY OF INFERENCE TECHNIQUES
Type of Inference Applicable Techniques
- Threat Analysis
- Situation Assessment
- Behavior/Relationships of Entities
- Identity, Attributes and Location of an Entity
- Existence and Measurable Features of an Entity
High
Low
- Knowledge-Based Techniques
- Decision-Level Techniques
- Estimation Techniques
- Signal Processing Techniques
- Expert Systems- Scripts, Frames, Templating- Case-Based Reasoning - Genetic Algorithms
- Neural Nets- Cluster Algorithms- Fuzzy Logic
- Bayesian Nets- Maximum A Posteriori
Probability (e.g. Kalman Filters, Bayesian)
- Evidential Reasoning
INF
ER
EN
CE
LE
VE
LIN
FE
RE
NC
E L
EV
EL
MULTI-LEVEL/MULTI-PERSPECTIVE INFERENCING
Level 1:Positional, Identity &Attribute Refinement
Level 2:SituationRefinement
Level 3:ThreatRefinement
Level 4:ProcessRefinement
where what whywhen who how
DATA FUSION PROCESSING
Level O:Signal Refinement
PHYSICAL OBJECTS
individual organizations
EVENTS
specific aggregated
TERRAIN & ENEMY TACTICS
local global
ENEMY DOCTRINE & OBJECTIVES
specific global
FRIENDLY VULNERABILITIES & MISSION
options needs
FRIENDLY ASSETS
specific & global
BENEFITS OF DATA FUSION: MARGINAL GAIN OF ADDED
SENSORS
Nahin and Pokokski, IEEE AES, 16 May 1980.
0.13
0.12
0.11
0.10
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.5 0.6 0.7 0.8 0.9 1.0
PN Single Sensor Probability of Correct Classification
P
N
(P
N +
2)
- P
N=
PN
PN
N=1(13 Sensors)
N=3(35 Sensors)
N=5(57 Sensors)
BENEFITS OF DATA FUSION: ENHANCED SPATIAL
RESOLUTION RADAR
TARGET REPORTLOS
ABSOLUTE UNCERTAINTYREGION INTERSECTION
RADAR ABSOLUTEUNCERTAINTY REGION
FLIR ABSOLUTEUNCERTAINTYREGION
ELEVATIONUNCERTAINTY
TARGETREPORT
COMBINED
AZIMUTHUNCERTAINTY
FLIR
TARGET REPORTLOS
TARGETREPORT
SLANT RANGEUNCERTAINTY
AZIMUTHUNCERTAINTY
SLANT RANGEUNCERTAINTY
ELEVATIONUNCERTAINTY
FLIR and Radar Sensor Data CorrelationFLIR and Radar Sensor Data Correlation
Adapted from W.G. Pemberton, M.S. Dotterweich, and L.B. Hawkins, “An Overview of Fusion Techniques”, Proc. of the 1987 Tri-Service Data Fusion Symposium, vol. 1, 9-11 June 1987, pp. 115-123.
THE DOD LEGACY: EXTENSIVE RESEARCH
INVESTMENTS• JDL Process model• Taxonomy of Algorithms• Lexicon• Engineering Guidelines
– Architecture Selection
– Algorithm Selection• Evolving Tool-kits• Extensive Legacy of
technical papers, books• Training Materials• Test-beds• Numerous prototypes
CHALLENGES IN DATA FUSION . . .
Robust sensors: no perfect sensors available difficult to predict sensor performance unable to effectively task geographically distributed non-commensurate sensors
Image and non-image fusion: no true fusion of imagery and non-imagery data unable to optimally translate image in time-series data into meaningful symbols no requisite models for coherent fusion of non-commensurate sensor data
Robust target identification: insufficient training data unable to perform automated feature extraction no unified method for incorporating implicit and explicit information for
identification (e.g., information learned from exemplars, model information, and cognitive-based contextual information)
CHALLENGES IN DATA FUSION . . .
Unified calculus of uncertainty (e.g., random set theory): do not know how to effectively use these techniques limited experience in trade-offs and use of fuzzy logic, rules probability,
Dempster-Shafer’s method, etc. unsure how to select the best uncertainty method
Pathetic cognitive models for Level 2 and 3: unknown how to select the appropriate knowledge representation
techniques argue about competing methods do not know how to use hybrid methods unable to perform knowledge engineering
CHALLENGES IN DATA FUSION . . .
Non-commensurate sensors: uncertainty as how to optimize use of wildly non-commensurate sensors inability to know how to link decision needs to sensor management unable to effectively use 10N sensors no consensus on MOE/MOP
Human computer interface (HCI): trendy and driven by technology and not cognitive needs of user suffer from the Gutenberg Bible syndrome no effective tools to overcome cognitive deficiencies unable to capitalize on built-in human pattern recognition (e.g., recognition of faces,
concepts of harmony)
DATA FUSION ISSUES . . .
What algorithms or techniques are appropriate and optimal for a particular application?
What architecture should be used (i.e., where in the processing flow should data be fused (viz. at the data, feature, or decision levels)?
How should individual sensor data be processed to extract the maximum amount of information?
What accuracy can realistically be achieved by a data fusion process? How can the fusion process be optimized in a dynamic sense? How does the data collection environment (i.e., signal propagation, target
characteristics, etc.) affect the processing? Under what conditions does multi-sensor data fusion improve system operation
(under what conditions does it impede or degrade performance)?
LESSON 1 ASSIGNMENTS
Review the on-line introduction material (lesson 1) Read chapter 1 of Hall and McMullen Writing assignment 1: write a brief biographical sketch of
yourself (to share with the class) Writing assignment 2: write a paragraph describing the
occurrence of data fusion in a natural setting Team Assignment (T-1) - Meet with your assigned team to
discuss the semester collaboration
DATA FUSION TIP OF THE WEEK
“Here’s where we plan to use data fusion.”
Despite enormous amounts of funding for data fusion research – there is still no magic data fusion system or techniques!