1
Beyond Human Factors: An Approach to Human/Automation TeamsHaomiao Huang Jerry Ding Wei Zhang Claire J. TomlinHybrid Systems LabAction Webs Meeting11/17/2010
2[nasa.gov, businessweek.com, tgdaily.com, techeasy.co.za, deere.com, aurore-sciences.org]
Advances in complex multi-agent systems require smart integration of human elements.
3
[foxnews.com] [wikipedia]
[media.weirdworm.com]
[knowyourmeme.com]
[adriandayton.com]
This requires new approaches to analyze humans as part of the system!
Let’s think about humans as part of the solution, not the problem.
4
Two related problems
2) Control - generating useful directives and controls for human agents
1) Modeling- Properly representing humans as components of the overall system
5
OutlineMotivation
Scenario for Research on Human/Automation Teams
Adversarial Game Problem
Reachability Based Approach
Results
Conclusions & Future Work
6
Choosing a Research ScenarioGames are representative of hard, real-world problems, yet provide relatively benign “sandbox” environments for development
Robocup
Chess
What is a good game to capture the aspects of human-automation teams that we want to explore?
Starcraft
Roboflag
7
Time tested and fun
Capture-the-FlagCapture-the-flag embodies the basic research challenges we are trying to address
http://www.goforyourlife.vic.gov.au/hav/articles.nsf/pages/Capture_the_Flag
Limited InformationMultiple AgentsCompeting Objectives
Human playersAdversarial
8
Automation-Assisted Human Capture-the-FlagUsing mobile phones, computers, and UAVs, we have turned capture-the-flag into a testbed for advanced automation concepts involving human team members
Game software on
Android phones
STARMAC Quadrotor
UAVs
Server-side Management Software
9
Time tested and fun
Narrowing the problem
http://www.goforyourlife.vic.gov.au/hav/articles.nsf/pages/Capture_the_Flag
Limited InformationMultiple AgentsCompeting Objectives
Human playersAdversarial
10
OutlineMotivation
Scenario for Research on Human/Automation Teams
Adversarial Game Problem Problem statement Related Work Solution Insights
Reachability Based Approach
Results
Conclusions & Future Work
11
Our Problem
Capture Region
Defender
Attacker
Flag
Flag Region
Return Region
Game Domain
Characterize and solve a 1-sided capture-the-flag game with a single attacker and defender
12
Related Work on Adversarial Games Multi-agent games on discrete state spaces
Greedy search
Hespanha, Kim, and Sastry 1999
Approximate DP/Reinforcement Learning
Lagoudakis and Parr 2002
Discrete Play Matching
Browning, Bruce, and Veloso 2005
Pursuit-evasion games with continuous statesReceding-Horizon Control
Mcgrew, How, Bush, Williams and Roy 2008
Sprinkle, Eklund, Kim, and Sastry 2004
Optimal Trajectory PlanningEarl and D’Andrea 2001
Chasparis and Shamma 2005
Analytical game theory approaches
Basar 1989, Lewin 1994,
Stipanovic, Melikyan, Hovakimyan 2010
Hamilton-Jacobi Reachability
Mitchell, Bayen, and Tomlin, 2005
Ding, Sprinkle, and Tomlin 2008
Assumed, learned, or randomized opponent model
13
ReachabilityApproach, derived from pursuit-evasion games: CTF game can be posed as a reachability problem.
Assume system dynamics
Where is the input for Player I and is the input for Player II
Define as the reach-avoid set where a player can arrive in a goal region in at most time while avoiding region , no matter what the other player does
14
Capture-the-Flag as ReachabilityVictory conditions for each player can be encoded as reach-avoid sets in the joint state-space
Defender
Attacker
Joint Capture Set
Joint Return Set
Flag Return Set (For Attacker)
Game Domain
15
1-D Game
16
Geometric insightsGeometric analysis allows some insight into the 2-D capture-the-flag problem
17
Geometric insightsGeometric analysis allows some insight into the 2-D capture-the-flag problem
18
Utility of Reachability AnalysisReachability analysis gives complete characterization of game, and are a natural display tool for guiding human decision-making and allowing least-restrictive control
Teo and Tomlin, 2003
Geometric analysis is not terribly general, though…
19
OutlineMotivation
Scenario for Research on Human/Automation Teams
Adversarial Game Problem
Reachability Based Approach Hamilton-Jacobi Reachability Computation
Results
Conclusions & Future Work
20
Hamilton-Jacobi ReachabilityReachability in continuous state-spaces can analyzed as a terminal cost-only optimization problem, solved backward in time
Reachability Cost Function
Classic Optimal Control Cost Function
Tomlin 2009
21
Level-Set RepresentationSets can be represented using sub-level sets of signed distance functions as terminal cost functions
Set operations using point-wise minimum and maximums can be used to create arbitrary sets
Tomlin 2009, Mitchell 2003
22
Solution Based on HJBI Equation
The cost-to-go function is the unique viscosity solution to the Hamilton-Jacobi-Bellman-Isaacs equation
Classic Optimal Control Cost Function
Hamilton-Jacobi-Bellman-Isaacs Equation
Optimal Hamiltonian
23
Reachability Via Modified HJBI Equation
The backward reachable set is the zero sub-level set of the viscosity solution to a modified HJBI equation
Modified HJBI Equation
Optimal Hamiltonian
Reachability Cost Function
Mitchell, Bayen, Tomlin 2005
24
Numerical Solution to the Modified HJBI Equation
The viscosity solution to the modified HJBI Equation can be computed on a grid using the Level Set Toolbox from UBC
http://www.cs.ubc.ca/~mitchell/ToolboxLS/index.html
26
OutlineMotivation
Scenario for Research on Human/Automation Teams
Adversarial Game Problem
Reachability Based Approach
Results HJBI Reachability applied to capture-the-flag Simulation results Experimental setup
Conclusions & Future Work
27
Problem Formulation for 1v1 Capture-the-Flag
HJBI reachability analysis allows us to fully characterize the game
Dynamics
Optimal Hamiltonian
Optimal Inputs
28
Flag Return & Flag Capture
Winning regions for each portion of the game can be calculated directly from reach-avoid conditions
29
Sequenced Capture and ReturnWinning regions for the full sequence (flag capture and subsequent return) can be computed by using the intersection of the flag return set and flag zone as the initial condition for flag capture
30
Simulation ResultsSimulation results demonstrate the use of the reachability solutions
31
Field Experiments in Progress
Reachability-based control and input directives are being implemented on Droid Incredible phones
Game software on Android phones
Server-side Management Software
Player Positions and State
Reachable sets & optimal control inputs
32
OutlineMotivation
Scenario for Research on Human/Automation Teams
Adversarial Game Problem
Reachability Based Approach
Results
Conclusions & Future Work
33
ConclusionsCapture-the-flag is great platform for developing human-
automation systems research.
A differential game formulation using HJBI reachability solves perfect information, 1v1 CTF
34
Future WorkWe have the “correct” answer to the adversarial problem… now what?
http://www.goforyourlife.vic.gov.au/hav/articles.nsf/pages/Capture_the_Flag
Limited InformationMultiple AgentsCompeting Objectives
Human playersAdversarial
35
Thank you!Questions?
Top Related