Data Processing for Classification - NAOC Processing for Classification Dean Keiswetter, Ph.D., ......

Data Processing for Classification

Dean Keiswetter, Ph.D., M.B.A.

Chief Scientist, Leidos Holdings Inc.

Data Processing – Objective

● Determine which anomalies, if any, result from buried

munitions

● The result of the analysis is a decision regarding the

nature of the source of the measured signatures

● The analysis should be:

Transparent

Quantitative

Documented

2

+

UX-Analyze =

Commercial mapping,

processing, &

visualization software for

earth sciences

0Ht

H

t

H 2

2

2

Induced EMI

ResponsePhysics

Principal Axis Polarizabilities

+

UX-Analyze =

Commercial mapping,

processing, &


earth sciences

0Ht

H

t

H 2

2

2

Induced EMI

ResponsePhysics


Analysis

Algorithms

UX-Analyze software (ESTCP MR-0910)

+

UX-Analyze =

Commercial mapping,

processing, &


earth sciences

0Ht

H

t

H 2

2

2

Induced EMI

ResponsePhysics


+

UX-Analyze =

Commercial mapping,

processing, &


earth sciences

0Ht

H

t

H 2

2

2

Induced EMI

ResponsePhysics


Metal Mapper

TEMTADS 5x5

TEMTADS 2x2

Classification

Sensors

Commercial

mapping,

processing, &

visualization

software

Data analysis algorithms embedded into

Geosoft’s Oasis montaj…

Software Overview

● UX-Analyze is fully integrated into Oasis montaj as a

menu driven set of functions for geophysical target

characterization and classification.

● These functions permit users to effectively classify buried

sources as Targets-of-Interest, or not.

● Released to the US Government and commercial

contractors (free of charge to recipients)

4

Topics

● Review Polarizations – the basis of the classification

decision

● Processing Fundamentals

1. Construct a Library (Expected Munitions and Clutter)

2. QC Measured Data (Blind and Background)

3. Invert and Look for Expected Munitions

4. Look for Unexpected Munitions

5. Prioritize

● Results and Final Products

5

Polarizations – the Basis of the Decision

6

Plan view of TEM

Dipole

Response Model

Location & Orientation +

Time (ms)

0.1 1 10

Po

lari

za

bili

ty

10-3

10-2

10-1

100

101

P1

P2

P3

Polarizations

Transients

from Rx

cube

Time (ms)

0.1 1 10

Pola

rizabili

ty (

m3/A

)

0.001

0.01

0.1

1

10

Polarizabilities

7

Time (ms)

0.1 1 10

Pola

rizabili

ty (

m3/A

)

0.001

0.01

0.1

1

Principal axis polarizabilities

completely describe EM response

of target

intrinsic to the target

invariant to burial depth or

target orientation

Topics

● Polarizations – the basis of the classification decision

● Processing Flow Fundamentals





5. Prioritize

● Results and final products

8

Construct a Library

QC Measured Data

Invert and Look for Expected

Munitions

Look for Unexpected Munitions

Prioritize

BTW - the processing flow applies to all

of the Advanced EMI Sensors

Advanced EMI sensors

are physically different,

but the extracted

polarizabilities are not...

9

100

100

sphere

100

100

105

100

100

60

100

100

small iso

100

100

37

100

100

sphere

100

100

105

100

100

60

100

100

small iso

100

100

37

Sphere 105 mm

60 mm Small ISO

time (ms) time (ms)

time (ms) time (ms)

MetalMapper

MPV

HandHeld BUD

TEMTADS 2x2

Construct a Library

QC Measured Data


Munitions


Prioritize

#1: Construct a Library

Document what we are Looking for…

Simply stated:

We want to specify which munitions are present at our

current site and store their respective polarizations in a library.

New items can be added to the list if encountered.

Why it is important:

Classification performance suffers if we look for munition

items that are not actually present…

10

Library Master

11

Database, maps

and scatter plot are

linked

Build a library for the munitions expected at your site. It

should include all anticipated munitions and unique

clutter items (if any).

Munitions Signature Examples

12

105mm 2.36inch

37mm

155mm

81mm

20mm

Munitions Variability

13

Variability among munitions

needs to be considered.

Site specific varieties,

whether anticipated or

discovered during the

program, must be added

to the library if unique…

Seeds can be Used to Verify Library

14

‘Manage Library Tool’:

• Checks adequacy of library…

• Identifies clusters…

Library

Overlay

Color coded

by number of

matches

0, 2, 5, 10, 20

Small ISO

Library

Current

Target Close

Matches

Small ISO Ellis TP1118

Construct a Library

QC Measured Data


Munitions


Prioritize

#2: QC Measured Data

Simply stated:

We need to examine the integrity of the measured data

files and the individual sensor readings


Garbage in garbage out…

15

QC Sensor Data for Blind Sources

Sensor…

Spatial

registration…

GPS

Orientation

data…

16

GPS Data

Inertial Measurement Unit

Evaluate Sensor Data bounds…take Action on Outliers

EMI Sensor Data

QC Data Backgrounds

17

Sensor data acquired

over areas believed to be

free of metallic objects

are subtracted to remove

sensor drift, biases, and

ground response

Background Example

18

R6 R7

Tx

Ty

Tz

R4 R5 R1 R2 R3

3 backgrounds are different from

the rest in varying degrees

1150 (orange) is most variable

followed by 1111 (green) and

1265 (red)

1150

1111

1265

Z_6Y

Low amplitude signatures are preferentially

affected by questionable backgrounds

19

37mm Result of subtracting the three

different backgrounds shown

earlier on a seeded 37mm…

1265 as background

37mm; 0.86 metric

1111 as background

37mm; 0.35 metric

1150 as background

No Match

Construct a Library

QC Measured Data


Munitions


Prioritize

#3: Invert and Look for Expected Munitions

Simply stated:

In this step, we want to invert the measured sensor data

to obtain polarizabilities and compare them against our

library of anticipated munitions.


Provides a quantitative comparison of intrinsic source

features

20

Primary ‘Turn the Crank’

21 Parameters

Process

Time (ms)

0.1 1 10

Po

lari

za

bili

ty10-3

10-2

10-1

100

101

P1

P2

P3

Location &

Orientation +

Inversion Process

Qualitative Visual-based Library Match…

Blue Unknown

Grey Library

from APG

60mm 81mm

105mm

Intr

insic

Pola

riza

tions

Intr

insic

Pola

riza

tions

0.1 1.0 10.0

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

10 0

10 1

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

10 0

10 1

0.1 1.0 10.0

0.1 1.0 10.0

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

10 0

10 1

105mm

Time (ms)

Time (ms)

Time (ms)

22

23

Parameters

Options

Multiple sources view

All Three Curves

Only Two Curves

Only One Curve

0.93

0.94

0.99

Match Type (ID)

Quantitative Library Match

UX-Analyze: QC environment…

24

Interactive review with multiple linked views

scatterplot, database, images, and polarizations

A mouse click in the scatterplot or database changes all…

User can zoom to see scatterplot details…

Cluster Library Match Data maps

Size / Decay

Database

Fit Results

BTW -- It ain’t that easy to

unravel issues…

25

Lots and Lots of Individual Steps… • Analysts need training and practice

• Third party QC review is critical

Construct a Library

QC Measured Data


Munitions


Prioritize

#4: Look for Unexpected Munitions

26

Simply stated:

We need to look for repeat source signatures (multiple

sources that are similar) that are not explained by the

library


We have to expect the unexpected. Historical records may

be incomplete. Site usage may vary by location…

Unanticipated UXO May Need to be Added

27

Library

Current

Target

Close

Matches

‘Manage Library Tool’:

• Checks adequacy of library…

• Identifies clusters…

Library

Overlay

Color coded

by number of

matches

0, 2, 5, 10, 20

Unanticipated TOI: Fort Sill, OK

?? – unique polarizations and best library match

28

29

3.5inch rocket

??

Unanticipated TOI: Fort Sill, OK

Fort Sill, OK – Ground Truth

40mm Frag Ball – no training data, unexpected

30

Construct a Library

QC Measured Data


Munitions


Prioritize

#5: Prioritization

31

Simply stated:

An anomaly is flagged for digging if the inverted source:

Matches a signature in the site-specific library of munitions

Is part of a previously unidentified cluster

Large and deeply buried


A numerical process produces a transparent,

quantitative, and consistent classification decision

Prioritization Decision & Validation Digs

32

Library Match Metrics:

Can be simplified to a

single threshold -- one

that is chosen to account

for all site specific UXO.

Possible approach for recommending Validation Digs:

#1: Interrogate x% of anomalies below selected threshold

#2: Analyst recommended checks

Parameters

Bounds set by controlled

tests & rules to deal with

distance-to-flag scenarios

This Section:

Not Site Dependent

Topics

● Polarizations – the basis of the classification decision

● Processing Fundamentals





5. Prioritize

● Results and Final Processing Products

33

Results and Final Processing

Products

34

Prioritized Dig List

The decision…

• Transparent

• Quantitative

• Documented

…and the documentation.

Digital files…

data maps, polarizations,

library comparisons, final decision

Dynamic Data Issues – Processing Perspective

35

Additional Tasks:

Pick Targets

Extract spatial data

Dynamic Data

36

Standard flow after

anomalies are selected…

Greater variability in

secondary polarizations, but

they are still sufficient to

identify TOI’s

Dynamic

Dynamic Stationary

Stationary

75mm

ISO’s

Construct a Library

QC Measured

Data

Invert and Look for Expected Munitions


Prioritize

37

Technology Transfer - Workshops

Year Location ½ Day

1 Day

2 Days Brief/Workshop # Attendees

3/2010 Huntsville, AL x workshop 32

4/2010 Denver, CO x brief 2

12/2010 Washington, DC x brief 40

1/2011 Washington, DC x workshop 21

1/2011 Denver, CO x workshop 21









TOTAL 306

Representatives from over 40 firms

Technology Transfer, continued

38

Results and Final Processing

Products

39

Prioritized Dig List

The decision…

Digital files

data maps, polarizations,

library comparisons, metric matches

…and the documentation.

• Transparent

• Quantitative

• Documented

Data Processing for Classification - NAOC Processing for Classification Dean Keiswetter, Ph.D., ......

Documents

Transcript of Data Processing for Classification - NAOC Processing for Classification Dean Keiswetter, Ph.D., ......