Standoff IED Detection Using UAVs

Week 1: Original Focus

Hardware - A small, portable drone:

● that uses different sensors to;

● replace human capabilities in detecting IEDs

Week 10: Final Focus

Software - A sense-making system:

● that uses image processing techniques to;

● augment human capabilities in detecting IEDs; and

● scales across platforms.

103 Interviews

PLOMOJIDOSponsor

103 Interviews

19 GovernmentAffiliated

12Image

ProcessingExperts

54 Armed Forces Personnel18

IndustryAffiliated

PLOMO

Yicheng An Weihan Zhang Robert André Borochok

Marko Jakovljevic

M.S. Computer Vision & Machine Learning

M.S. Business M.S. Management Science and Engineering

Postdoc ImagingSchool of Medicine

Software Engineering

Law Enforcement / Strategy & Operations

Industrial Engineering / Operations

Radiology / Image Processing

3

PLOMO

The Journey PLOMO

Hardware Software

Our Opening Hypotheses

It’s all about the Hardware

Multi-Functional Tool to Replace Human

Detects All Types of IEDs

PLOMO

Who’s our Beneficiary? … Everyone!PLOMO

Rock BottomPLOMO

PLOMO

Our problem scope is far too wide.

We Need to Truly Understand the ProblemPLOMO

“Infantrymen are the most vulnerable to IED attacks...they are trained to detect IEDs, but rely mainly on visual cues.”

- Operational Commander

PLOMO

Focusing on Our Primary Beneficiary

PRIMARY: DISMOUNTED

INFANTRY

Most vulnerable to attack

Relies on visual cues (potholes, disturbed earth)

Needs standoff detection capability

Load is a major constraint

Needs near real-time detection

11

PLOMO

Week 3: Mission Model Canvas

Value Proposition

Replace infantrymens’ capabilities to detect IEDs

A drone system

System to analyze drone feed and indicate risk areas to war fighter

Beneficiaries

Primary: Dismounted infantry patrolling known area

12

PLOMO

Let’s build a counter-IED drone for ground infantry...

PLOMO

...Or not.

“We already use surveillance drones…why would we need another one?”

- Operational Commander

PLOMO

Let’s Modify Existing Hardware!

Build a Drone Add Sensors to Existing Drones

PIVOT

PLOMO

Pivot to Sensor ImprovementsPLOMO

Hypothesis:Sensors added to drones like Raven will increase situational awareness and ease of IED detection.

Reality:Sensors on their own do not give analytical insights

Pivot to Sensor ImprovementsPLOMO

Hypothesis:Sensors added to drones like Raven will increase situational awareness and ease of IED detection.

Reality:Sensors aren’t perfect

“There is no vapor for chem sensors to sniff in an open environment.”

- Explosive Signature Specialist

Pivot to Sensor ImprovementsPLOMO

Hypothesis:Sensors added to drones like Raven will increase situational awareness and ease of IED detection.

Reality:Adding sensors will require long deployment time

“JIDO can’t … add a sensor to a program of record.”

- JIDO Tech Chief

Hardware is Clearly not the IssuePLOMO

“My analysts spend hours staring at a screen to pick up anomalies. We still miss things from time to time.”

- Intelligence analyst

PLOMO

Getting out of the buildingPLOMO

Camp Pendleton (San Diego)

Because of the near infinite number of ways an IED can be hidden, we limit our initial product to pothole detection.

PLOMO

Week 5: Mission Model Canvas

23

Value Proposition

Augment replace

warfighters’ capabilities

to detect IEDs via software

Drone agnostic

Pothole Detection

PLOMO

Software is What’s Really Needed!

Drone Hardware “Analyst in the hand”

PIVOT

PLOMO

We Have Found Xplomo’s CallingPLOMO

Our MVP and Beneficiary Needs Match.

So Let’s Start Identifying Partners.

PLOMO

So Who Do We Need to Work With?PLOMO

So Who Do We Need to Work With?PLOMO

XPLOMO

Week 6: Mission Model Canvas

Key Partners

JIDO J8/J6

Image Processing Experts

Mission Achievement

- Algorithm that successfully detects potholes with a false alarm rate <5 per frame

- At least 80% accuracy

Buy-in/Support

JIDO

Troops in field

Military Leadership

Mission Achievement

- Algorithm that successfully detects potholes with a false alarm rate <5 per frame

- At least 80% accuracy

In the rush to develop a working product, you’ve got to

Fail Fast, Move Quick

PLOMO

Feature Detection Works…. ...Until It Doesn’t

PLOMO

...

Xplomo’s Experimental Results

PLOMO

Academic Articles and 1 Paper Later...

Feature Detection

Anomaly Detection

Machine Learning

PLOMO

“Analyst in the hand”

Successful Detection of Well-Defined Potholes

Xplomo’s Experimental Results

Week 8: Mission Model Canvas Key Activities

Implement image processing methods:

● Anomaly detection -CFAR

● Machine learning -YOLO

Train the algorithms

Package the software in JIDO compatible format

PLOMO

Deployment

Algorithm refinement, Scaling, & Horizontal extension

Final Minimum Viable Product

“Analyst-in-the-hand” Operator

Any Aerial Platform

PLOMO

Internal Readiness LevelPLOMO

Current product is an initial

iteration of an algorithm that we

will continue to improve.

Internal Readiness Level

Moving Forward

Buried IEDs

VBIED

Week 1 Week 10

H4D

MilitaryGIS Output

Dual use

Agriculture

Forestry

Road maintenance

Building inspections

PLOMO

Thank YouJIDOWayne A. StanberyW. Richards Thissell James McGuyer

Industry MentorKevin RayRobert Medve

Special ContributorsRobert BestCaitlin Cima Todd Forsman Andrea GilliGeorge Hasseltine Rafi Holtzman Michael LeoneDavid Zinn

Special ThanksCamp Pendleton Commanders and Staff& to all our 103 Interviewees

38

Teaching TeamSteve BlankJoseph FelterPeter NewellSteve Weinstein

Teaching AssistantsDarren HauIsaac MatthewsMelisa Tomak

PLOMO