Dynamic Data Driven Applications System concept for Information Fusion Erik Blasch Guna Seetharaman,...

Dynamic Data Driven Applications System concept

for Information Fusion

Erik Blasch

Guna Seetharaman, Kitt Reinhardt [email protected]

PM: Dr. Frederica Darema

International Conference on Computational Science June 04-05 2013

Presented by Craig Douglas

mailto:[email protected]

Blasch/Seetharaman – ICCS 13

OUTLINE

• Motivation

• DDDAS Concept for Wide-Area Motion Imagery (WAMI)• DDDAS match with Information Fusion

• DDDAS rich with applications to complex environmental modeling

• Information Fusion – Surveillance, Target Tracking• Developments in motion imagery with large-data formats

• Focused on target tracking : need environment model in WAMI

• DDDAS processing for WAMI• Statistical Mathematical modeling

• Developing results over different applications

• Summary: Future reporting in 2014 WAMI text


DDDAS Mathematics ModelingMultimodal Cooperative Sensing

• Motivation• Using the DDDAS concept (mathematics, modeling, and software);

information fusion systems composing multi-modal sensor measurements can be enhanced to consider current trends in big data (e.g. large imagery), enterprise architectures (e.g. dynamic), and systems management for real-time cooperative sensing.

• The novelty of the work consists of developing a statistically optimal image perspective formation using 3D homographical mappings applied to Wide Area Motion Imagery for scene characterization, target tracking, and situational awareness.

• Applying the advanced scene characterization software in the AFRL PCPAD-X (Planning & Direction, Collection, Processing & Exploitation, Analysis & Production, and Dissemination eXperimentation) Program - that consists of multi-intelligence data fusion from streaming full-motion video (IMINT - Image) and operator textual reporting (HUMINT – human), the emerging situation can be understood for real-time mission management.


DDDAS (Multimodal Cooperative Sensing)

Dynamic Data Driven Application Systems (DDDAS)

Theory

Simulations

Measurements

Sensor Management

Forecasting, Prediction

Operational Condition Fidelity

Filtering

Mission Management

User Refinement

Situation Assessment

Object ID and Tracking

Full Motion Video

Modeling

Multi-scale Multimodal Data

2000 2500 3000 3500 4000 45000.996

0.997

0.998

0.999

1

1.001

1.002

1.003

1.004x 10

4


DDDAS Background

• Other works of interest• DDDAS since its inception has been applied to numerous areas where complex

real-world conditions are not predetermined by initialization parameters and data• Darema, F., 2004. “Dynamic Data Driven Applications Systems: A New Paradigm for Application

Simulations and Measurements. Computational Science

• DDDAS is related to Information Fusion in processing stochastic information • Douglas, C. C., Efendiev, Y., Ewing, R., Ginting, V., and Lazarov, R., 2006. “Dynamic Data Driven

Simulations in Stochastic Environments,” Computing, 77(4):321–333.

• DDDAS applied to Environmental modeling, Situation awareness, and Systems-level applications

• Environmental modeling• Oceans [Patrikalakis, 2004], Wildfires [Chen M, 2005]

• Transportation [Fujimoto R. M; 2004], emergency medical response [Gaynor M; 05]

• Waste distribution [Parashar, M, 2006; Mahinthakumar, K, 2006].

• Collaborative Adaptive Sensing of the Atmosphere (CASA) (Radar) formed by the National Science Foundation [Brotzge, J., 2006].


OUTLINE

• Motivation









Information Fusion and DDDAS

• DDDAS• Supports large (complex) data processing

• Reasons over stochastic uncertainty

• New Application: Enterprise Intelligence, Surveillance, and Reconnaissance

• Information Fusion• Sensor Fusion (sensing)

• Info Fusion (enterprise)

• User Interaction (Report)


Information Fusion and DDDAS

• DDDAS and Information Fusion• Environmental modeling for object

assessment, situation and impact assessment over mission needs

E. P. Blasch, E. Bosse, and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, MA, 2012.

• Information Fusion• Processing Levels :

L0 data registration, L1 object assessment,

(tracking, classification)L2 situation awareness L3 impact assessment

(threat). L4 process refinement, L5 user refinement L6 mission management

• Applications : emergency response, sensor /user control (CASA), transportation. HERE, focused on ISR tracking in imagery


Point of View

Cover Proc IEEE, Nov 2010


OUTLINE

• Motivation









CLIF 2006/2007

EO UAVMWIR UAV LAIR

Building mounted EO

From Olga Mendozza-Schrock, Jim Patrick, E. Blasch, “Image Techniques for Layered Sensing,” IEEE NAECON09.

Public Release No. RY-09-0449


DDDAS Mathematics ModelingMultimodal Cooperative Sensing

f(x, y; t) f(x, y; t + 1)

Surface (S1)(e.g. tennis court)

Line (S2)(e.g. telephone pole)

Wall (S3)(e.g. fence)

Cube (S4)(e.g. building)

COMPLEX(e.g. City)

• Complex Scene Analysis • Data: Text (analyst reports) - “Find Building A”

• Data: Geospatial (terrain) - “3D surface”

• Dynamic: Perspective change from moving camera

• Surface Mapping• Simple: 2D surface homography

• Complex: Different surfaces in 3D

• Least Squares corrupts 3D result

• Science Contribution• Locate primitive surfaces

• Determine dynamic change through mixture of piecewise optimize mappings

DDDAS


DDDAS Applications Modeling (1)Multimodal Edge Detection

WAMIWide-Area Motion

Imagery

Road

EW Road

[ℓ x, ℓ y, ℓ z]

WAMI Modeling

• Complex Imagery

• Changing Dynamics

• Model testing, and update

• Decisions –

• Terrain Information (environment)

• Man-made objects

(Buildings, targets)

EnvironmentVanishing Point

DDDAS

WAMI MODEL

Vanishing Point

(ℓx , ℓy)

H-Test

H0 / H1Edges


DDDAS Applications Modeling (2)Multimodal Edge Detection

SensorVanishing Point

WAMIWide-Area Motion

Imagery

nadir

zenith

South north

West East

Z w

y c

X w

Y w

x c

z c

e y

e z

Road

EW Road

[ℓ x, ℓ y, ℓ z]

Multimodal Inputs (only WAMI displayed)

• Text (analyst reports)

• Geospatial (terrain)

• Use primitives (through vanishing points)

Multimodal Difficult

Text: “Find building A”

EnvironmentVanishing Point

TargetVanishing Point

DDDAS


DDDAS Mathematics and Stat ModelingMeasurement based Optimality

Non-Homography relation Transforms (between multimodal collections)

f(x, y; t) f(x’, y’; t + 1)

Surface (S1)(e.g. tennis court)

Line (S2)(e.g. telephone pole)

Wall (S3)(e.g. fence)

Cube (S4)(e.g. building)

COMPLEX(e.g. City)

Homogeneous

z = - px – qy - s

c s a

= { S1, S2, S3, S4}

s = each surface

L34 = S3 S4

= { S3, S4}

Mathematical Issues Working• Must have tangential conformity• Optimality Conditions (from terrain)• If Imagery, how other sensors• Joint DDDAS optimization

L = direction cosine

DDDAS


DDDAS Systems SoftwarePCPAD-X (Processing, Collection, and Dissemination)

4 Collaboration: RY (Sensors), RI (Information), RH (Human Effectiveness)

Fusion Testbed

Evaluation

FMV Terrain

User Visualization of complex scene

Evaluation

User (filter,

analyze, report)

Data Exploitation

Multi-INT Fusion

Viewer

DDDAS

Multiscale chat, text, doc

Multiscale imagery


OUTLINE

• Motivation








Dynamic Data Driven Applications System concept for Information Fusion Erik Blasch Guna Seetharaman,...

Documents

Transcript of Dynamic Data Driven Applications System concept for Information Fusion Erik Blasch Guna Seetharaman,...