SingleOperatorControlofMul2pleUAS:A

SupervisoryDelega2onApproach

PresentedtoUASEXCOMScienceandResearchPanel(SARP)WorkshoponSingleOperatorControlofMul2pleUAS

JayShivelyNASA-AmesResearchCenter

UAS INTEGRATION IN THE NAS

JayShivelyDAASub-ProjectManager

1

Levelsofautoma2onofDecisionandAc2onSelec2on(Sheridan&

Verplanck,1978)

1Thecomputeroffersnoassistance,humanmusttakealldecisionsandac2ons2Thecomputeroffersacompletesetofdecision/ac2onalterna2ves,or3Narrowstheselec2ondowntoafew,or4Suggestsonealterna2ve,and5Executesthatsugges2onifthehumanapproves,or6Allowsthehumanarestrictedveto2mebeforeautoma2cexecu2on7Executesautoma2cally,thennecessarilyinformsthehuman,and8Informsthehumanonlyifasked,or9Informsthehumanonlyifit,thecomputer,decidesto10Thecomputerdecideseverything,actsautonomously,ignorestheHuman

SupervisoryControlSheridan(2002)definedsupervisorycontrolasanarrangementinwhich“oneormorehumanoperatorsareintermiaentlyprogrammingandcon2nuallyreceivinginforma2onfromacomputerthatitselfclosesanautonomouscontrolloop,”buthealsoaccentuatedthehumansystemrela2onshipunderlyingthedefini2on:“Supervisorycontrolderivesfromthecloseanalogybetweenasupervisor’sinterac2onwithsubordinatepeopleinahumanorganiza2onandaperson’sinterac2onwithintelligentautomatedsubsystems”Supervisorycontrolisageneraltermforcontrolofmanyindividualcontrollersorcontrolloops,suchaswithindistributedcontrolsystem.Itreferstoahighlevelofoverallmonitoringofindividualprocesscontrollers,whichisnotnecessaryfortheopera2onofeachcontroller,butgivestheoperatoranoverallplantprocessview,andallowsintegra2onofopera2onbetweencontrollers.

Delega2onControl:Playbook®•  Delega2on:onewayhumansmanagesupervisory

controlwithheterogeneous,intelligentassets

•  Playbook®:onesmeansofdelega2on

•  Plays:analogoustofootball–  Quickcommands–complex

ac2ons

•  APlayprovidesaframework–  Referencesanacceptablerange

ofplan/behavioralterna2ves–  Requiressharedknowledgeof

domainGoals,TasksandAc2ons–  Supervisorcanfurtherconstrain/

s2pulate•  Poten2allyfacilitatesintui2vecoopera2vecontrolof

UnmannedSystems

•  Drill-downandmodifyasrequiredbycontext

ApagefromAlonzoStagg’s1927Playbook

TANGO

Predator Provides Overwatch and Hellfire strike capability

Shadow designates target

Firescout Does Quick Med Drop

Example:TroopsinContactTango

Example:ProsecuteTarget

Tools:Armlaser➔Lasetarget➔SendcoordinatestoweaponizedUAV➔ToggleUAVs➔Armmissile➔Fire

Scripts:Select‘Lase’script➔ToggleUAVs➔Armweapons➔Fire

Plays:Select‘ProsecuteTarget’play➔Fire

0%

10%

20%

30%

40%

50%

60%

70%

80%

Plays Scripts Tools

% C

orre

ct

Control Mode

Secondary Task Performance (% Correct Hits)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Plays Scripts Tools

Ove

rall

Rat

ing

Control Mode

NASA-TLX Ratings

0

10

20

30

40

50

60

Plays Scripts Tools

Rea

ctio

n Ti

me

Control Mode

Primary Task Performance (RT)

Target Acquisition

Target Prosecution

ShorterReac2onTimeforPlays

PlayshadlowerworkloadHigherAccuracyforPlays

LevelsofAutomaConSimulaCon

Manned-UnmannedTeaming:MUM

Level IV Control: Control of Payload and Vehicle Excluding Take-off and Landing

Extend to simultaneous control of multiple heterogeneous UAS

Manned-UnmannedTeaming:MUM

Goals:

•  ApplyPlaybook®methodologyandDelConlessonslearnedtohelicoptercockpit;Testinsimula2on

•  IncreasecapabilityandefficiencyofUAScontrolbyhelicopterpilots

•  Supervisorycontrolofmul2ple,heterogeneousUAS

•  Developinfrastructureandlayfounda2onforlaterefforts

Propor2onofTargetsMarkedbyControlMode(OutofTotalPossible)

0

0.2

0.4

0.6

0.8

1

NoUAV Manual Playbook

Prop

orCo

n

ControlMode

p < .05

0

5

10

15

20

25

Manual Playbook

Time(s)

ControlMode

UASRoutePlanningTimebyControlMode

Results

0

2

4

6

8

10

Temporal Frustra2on Performance Overall

AverageRa

Cng

WorkloadDimension

NoUAV Manual Playbook

NASA–TLXRa2ngs

p < .05 p < .05 p < .05

p < .05

HigherAccuracyPlaybook

LowerRoutePlanningTimeforPlaybook

LowerworkloadforPlaybookonseveraldimensions

FlightDemonstra2on2009Ft.OrdCA,23APR2009

Goal:•  Demonstratesini2alproofofconceptof

Delega2onControl(Playbook)inflight–supervisorycontrolofmul2pleair/groundassetsinMOUTScenario

Method:•  Live/VirtualDemo–ControllingRMAX,CMU

MAXRoverand2virtualUASwithDelega2onControl

•  VoiceRGNControl(USAF)

Features:•  Delega2oncontrolhuman-machineinterface

supportscontrolandmonitoring4payloads•  Automa2onTransparency•  LiveUGV-UAVcoordina2onforslungload

drop•  Reducedoperatorworkload/highsitua2on

awareness

Live RMAX Virtual Shadow

Virtual Sky Warrior

Live CMU

MAX Rover

•  Troopsincontact

•  RouteRecon

•  AreaRecon

•  Convoysupport

•  QuickMeds

TopPlays

FlightDemonstra2on2011Ft.Hunter-Ligge9CA,19May2011Purpose:•  Buildonprevioussimula2onsandflighttest

examiningsingleoperatorcontrolofmul2pleheterogeneousground/airunmannedsystemsthroughdelega2oncontrolemployment–  Operatorperformancedatacollec2on/workload

assessments–  Heterogeneousflightassets:BoeingScanEagle

andYamahaRMAX;twovirtualUAS–  Tes2nginopera2onallyrelevantmissionscenarios–  Mul2-sensorcross-cueinsupportofboth

targe2ngandconvoysupport•  ArmyAFDD/BoeingCRADAKeyObjec2ve:•  DevelopandtestDelConTopPriorityPlays;

routerecon,convoysupport,troopsincontact

Demonstratedinnumeroussimula2onsandflighttests(evenNOPEsimula2ons)

•  AFRL–Basesecurity,UASgroundsta2on

•  RCO–Dispatch,cockpit

•  HumanAutoma2onTeaming(HAT)

SupervisoryControlSummary

13

CivilUASPlays

•  MonitorBorder– Flydesignatedborder– Alertany“signsoflife”

•  UAS1–flywaypointatob•  UAS2–flyWPbtoC•  UAS3–follow-upwithanyalerts

•  Evaluatepowerlines•  Transitairspace

CivilPlays

•  SearchandRescue– Flydesignatedareasofsearchzone–lawnmowerpaaern,alertshapes,colors,etc.

– Survivaldrop–assoonasWPisdesignated•  Meds•  Radio•  Food/water•  Shelter

HATAgent

HAT Agent

Operator

Interface

DisplayAudioVisual

Context - Time Pressure- User Info- more

Queries/Requests - A v. B- Why?- What If?

Automation

Plays- Goals- Risks to

achieving goals- Mitigations

AlertsContextResponses to Queries- Alternatives- Transparency info - Predicted Outcomes - Reasoning - Confidence level

Transparency Info

RequestsPolling for Risks

Etiquette Rules/Contextual Sensitivity

Authority Info

Scratch Pad

16

•  Pilotdirectedinterface

–  Nointentinferencing–  Directedbypilotac2ons–  Nosetrolesandresponsibli2es–  Playbook

•  Bi-direc2onalCommunica2on

–  Why?–  Howconfident?–  Whatif?–  Addinforma2on

•  Transparency

–  Calibratedtrust–  Granularity–  Timepressure

HATAaributes

17

ProblemswithAutoma2on

•  Briale–  Automa2onotenoperateswellforarangeofsitua2onsbutrequireshuman

interven2ontohandleboundarycondi2ons(Woods&Cook,2006)

•  Opaque–  Automa2oninterfacesotendonotfacilitateunderstandingortrackingofthe

system(Lyons,2013)

•  MiscalibratedTrust–  Disuseandmisuseofautoma2onhaveleadtoreal-worldmishapsand

tragedies(Lee&See,2004;Lyons&Stokes,2012)

•  Out–of-the-LoopLossofSitua2onAwareness–  Trade-off:automa2onhelpsmanualperformanceandworkloadbut

recoveringfromautoma2onfailureisotenworse(Endsley,2016;Onnasch,Wickens,Li,Manzey,2014)

18

HATSolu2onstoProblemswithAutoma2on

•  Briale–  NegoCateddecisionsputsalayerofhumanflexibilityintosystembehavior

•  Opaque–  Requiresthatsystemsbedesignedtobetransparent,presentraConaleand

confidence–  Communica2onshouldbeintermstheoperatorcaneasilyunderstand

(sharedlanguage)

•  MiscalibratedTrust–  Automa2ondisplayofraConalehelpshumanoperatorknowwhentotrustit

•  Out–of-the-LoopLossofSitua2onAwareness–  Userdirectedinterface;adaptable,notadap2veautoma2on–  Greaterinterac2on(e.g.,negoCaCon)withautoma2onreduceslikelihoodof

beingoutoftheloop

19

WorkingAgreements

•  Pre-determinedauthoritysharingagreementswithautoma2on–  Ifthewatercoolingleveldropsbelowacertainvalue,openvalvestoemergencycooling

•  Autonomy– Notmuchintoday’s“approved”UAS– WordsMaaer

•  ICAO

•  Businesscaseforsingleoperatorsupervisorycontrolofmul2pleUAS–  Playbookdelega2onisonesuccessfulmethod

•  HAT–  Coopera2veagentwithknowledgeofworkdomain–  Sharedworldknowledge–  Canbeextendedtonetworksupervision

Summary

21