Joel Predd and Henry Willis February 26, 2009
description
Transcript of Joel Predd and Henry Willis February 26, 2009
Joel Predd and Henry WillisFebruary 26, 2009
Toward Adaptive, Risk-Informed Allocation of Border Security Assets
2
RAND Research on Counter-IED Operations in Iraq Illustrates Benefits of Tools
• Problem: Ground forces in Iraq had limited resources for counter-IED operations
• Method: RAND developed methods and tools to predict location and time of future IED threats based on database of recent attacks
• Application: Threat predictions helped brigades decide where to direct surveillance
W. Perry and J. Gordon, “Analytic Support to Intelligence in Counterinsurgencies”, RAND MG-682-OSD, 2008.
3
The Problem Concerns Operational Resource Allocation
U.S. law enforcement agencies need to direct limited border resources to detect and identify risks along the border
4
This Problem Statement Includes Four Key Terms That Need to be Further Defined
U.S. law enforcement agencies need to direct limited border resources to detect and identify risks along the border
• Resources include both technology and people
• Focus on resources that detect and identify, enable engagement and resolution
• Potential risks include both smuggling and border crossing
• Southwestern land border is the near-term focus, plan for extensions to North
5
Study Objective
To develop and evaluate machine learning-based methods and tools to facilitate adaptive, data-driven, risk-based allocation of border security resources
6
Four Principles Guide The Study Objective
To develop and evaluate machine learning-based methods and tools to facilitate adaptive, data-driven, risk-based allocation of border security resources
• Machine learning refers to a set of statistical and computational methods
• Method should– be adaptive, because border
crossers are
– be informed by data
– incorporate border threats, vulnerabilities and consequences (i.e., risk)
7
Example 1: Allocating Counter-IED Surveillance Assets
• Problem: Ground forces in Iraq had limited resources for counter-IED operations
• Method: RAND developed methods and tools to predict location and time of future IED threats based on database of recent attacks
• Application: Threat predictions helped brigades decide where to direct surveillance
W. Perry and J. Gordon, “Analytic Support to Intelligence in Counterinsurgencies”, RAND MG-682-OSD, 2008.
8
Example 2: A Meta-Allocation of Problem of Choosing Predictive Tools
• (Meta-)Problem: Ground forces in Iraq had to choose one of multiple predictive tools
– Each tool was itself designed to facilitate surveillance resource allocation, and better in different circumstances
• Method: RAND developed online learning methods to adaptively aggregate suite of tools based on historical performance
• Application: Aggregate tools could support original surveillance asset allocation problems
W. Perry and J. Gordon, “Analytic Support to Intelligence in Counterinsurgencies”, RAND MG-682-OSD, 2008.
9
Example 3: Research at USC CREATE Provides Another Illustration
• Problem: Airport security has limited resources to allocate to checkpoints and canine patrols
• Method: Researchers at USC CREATE developed methods and tools to systematically schedule checkpoints and canine patrols based on theory of Bayesian Stackelberg games
• Application: Software tool called ARMOR is used to schedule canine patrols
Pita, J., Jain, M., Western, C., Paruchuri, P., Marecki, J., Tambe, M., Ordonez, F., Kraus, S., Deployed ARMOR, "Protection: The Application of a Game Theoretic Model for Security at the Los Angeles International Airport," in Proceedings of the Seventh International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS) (Industry Track), 2008
10
We Are Working to Leverage This Research to Benefit CBP Operations
• Limited resources require tactical decisions about how to allocate
– Ground sensors– Patrols– UAVs– Detection – …
• How to do so in way the adaptively integrates tactical data about threats, vulnerabilities and consequences?
11
The Product: A Tool To Help Sector Chiefs Deploy Sensors and Patrols According to Risk
• The tool will identify future risks by making predictions from historical data
• Threat data
– E.g., data may include a record of the location and time of past detections or interdictions
• Vulnerability data– E.g., GIS data about cross-border roads or paths, sector boundaries
– E.g., GIS data about topography and weather
– E.g., Location and time records of previous border security operations, sensor deployments, and patrols
• Consequence data
– E.g, information on mission-types
12
Methodology and Work Plan
Year 1: Understand border operations, environment, and available intelligence data and collection assets
Year 2: Evaluate machine learning-based methods in a simulated environment
Year 3: Explore with CBP interest in conducting field evaluation of prototype tools
• Plans to visit San Diego Sector- Operation Red Zone- Border Intelligence Center- Air and Marine Operations Center
• Plans to visit Rio Grande Valley Sector
13
Summary
•A project funded through the National Center for Border Security and Immigration
•The objective is to develop and evaluate predictive methods and tools to facilitate adaptive, data-driven and risk-based allocation of CBP assets
•The outcome will be that Office of Border Patrol and the Secure Border Initiative program office will have methods and tool to dynamically allocate assets in the tactical environment
15
The Tool Automatically Identified Actionable Hot Spots of Enemy Activity
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
Past IED event
Road
16
The Tool Automatically Identified Actionable Hot Spots of Enemy Activity
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused Hot spots
5 miles
17
The Tool Automatically Identified Actionable Hot Spots of Enemy Activity
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
ActionableHot spots
500 meters
18
The Tool Automatically Identified Actionable Hot Spots of Enemy Activity
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
Highest ranking actionable hotspotswere candidates for surveillance
500 meters
Problem: CBP and local law enforcement need to direct limited border resources to where they can most effectively detect and identify risks along the border
Objective: To develop and evaluate machine learning-based methods and tools to facilitate adaptive, data-driven and risk-based allocation of CBP resources
Methodology
Phase 1: Field studies to CBP sites to understand border operations, environment, and available intelligence data and collection assets
Phase 2: Develop machine learning-based methods and prototype tools, and evaluate them in a simulated environment
Phase 3: Field studies to deploy prototype tools
Benefits to DHS
The Office of Border Patrol and the Secure Border Initiative program office will have tools to dynamically allocate assets in the tactical environment
Deliverables and TimelinesQ1, Q2, Q3 : Visit DHS, CBP Sites; review literature; Q4: Document findings
Year 1 Deliverables: Inventory of available intelligence assets; assessment of available data via whitepaper
Year 2 Deliverables: Method, prototype tool, and evaluation
Year 3 Deliverables: Assessment of field studies 19
NC-BSI: Adaptive, Risk-Informed Resource Allocation
Elevator speechTo manage the risk of illegal border crossings and smuggling, CBP must answer two resource allocation questions: Where and when should we conduct surveillance? Given the adaptive behavior of border crossers answering these questions requires an adaptive, data driven approach. This project will develop and evaluate such an approach.
Ongoing/leveraged research
JIEDDO-funded RAND IED research– Tactical support– Analysis of Alternatives
Risk analysis work with USC-CREATE– ARMOR and Border Risk Model
Costs and Special Equipment
Year 1: $77,250Year 2: $87,300Year 3: $90,000
Investigators
Henry H. Willis, Ph.D.Joel Predd, Ph.D.
20
NC-BSI: Adaptive, Risk-Informed Resource Allocation
21
RAND Analysis Uses Models and Simulations To Support Operational Integration
Virtual M&S
ComputationalModels
Field (Live)M&S
ConstructiveM&S
Iterativeprocess
22
We Are Seeking Guidance on Three Topics• What operational constraints must we take into account
– Visit border sites• Operation REDZONE, JTF-North Campaign Planning Workshop, El Paso
Information Center, Air and Marine Operations Center– Discuss CBP operations at sectors
• Recommendations related to scope of focus– Which sector(s) or station(s) to visit?– Which tactical operations might benefit most?– Who to meet? Where to visit?
• What sample data is available?– Location and time of past detections, interdictions– Location and time of past operations, sensor deployments, and patrols – GIS data about border roads, paths, topography, weather, etc.– After Action Reviews (AARs)
23
Study Plan is to Build Tools That Integrate With Current Practices
• We have learned that sectors may use different methods, and possibly share data and lessons learned
• Southwest sectors have employed some predictive methods for resource allocation
• Data about the location and time of some border activities are archived, shared
Source: Operation Gulf WatchProvided By: PAIC Mark Butler, Fort Brown Station, RGV SectorProvided To: MAJ Eloy Cuevas, JTF-N Intelligence Planner Date: February 2006
24
RAND Research on Counter-IED Operations in Iraq Illustrates Benefits of Tools (Example 2)
• Problem: Intelligence had developed many predictive tools, but had difficult choosing which heuristic to use for resource allocation
• Method: RAND developed methods to adaptively aggregate large suites of predictive tools using online learning
• Application: The aggregate tool provided a way to make a useful tool out of many
W. Perry and J. Gordon, “Analytic Support to Intelligence in Counterinsurgencies”, RAND MG-682-OSD, 2008.
25
Example 1: Allocating Counter-IED Surveillance Assets (2/3)
• Hot spot – an area consistently and recently targeted by enemy forces
Hot spots
5 miles
26
Example 1: Allocating Counter-IED Surveillance Assets (3/3)
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
ActionableHot spots
500 meters
27
Example 1: Allocating Counter-IED Surveillance Assets (3/3)
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
Highest ranking actionable hotspotswere candidates for surveillance
500 meters
28
Example 1: Allocating Counter-IED Surveillance Assets (3/3)
• Hot spot – an area consistently and recently targeted by enemy forces
• Actionable hot spot – a hotspot where limited surveillance resources can be focused
Highest ranking actionable hotspotswere candidates for surveillance
500 meters
The main success of this research was the integration of predictive methods
with operational constraints
29
Example 2: A Meta-Allocation of Problem of Choosing Predictive Tools (2/3)
• Predictive heuristics admitted essentially no theoretical analysis of effectiveness.
• Existing empirical analyses are optimistic; the results generalize only if the methods are not actually used in the field.
– in practice, enemy reacts to allocation methods use of a method; existing data does not reflect adaptation
• Long-term trends and normal reactive behaviors can go undetected.
time
location
…
30
Example 2: A Meta-Allocation of Problem of Choosing Predictive Tools (3/3)
• RAND developed online learning algorithms to adaptively aggregate a suite predictive tools
• Algorithms have provable performance guarantees
• Laboratory experiments suggest competitive to rival methods Day
Cu
mu
lat i
ve los s