AUV Workbench and Autonomous Vehicle Command

Language (AVCL)Progress and plans

Don Brutzman and Duane DavisNaval Postgraduate School

18 October 2011

AUV Workbench poster

AUTONOMOUS UNMANNED VEHICLE (AUV) WORKBENCH

Planning, generating and running robot missions

4

Our 3 R’s: rehearsal, reality, replay

Same needs and capabilities for each: mission, visualization, data support, etc.

AUV workbench supports each• ongoing work, starting to mainstream

15 years of accumulated effort• integrating great variety of successful work• new work projects occurring regularly

Collaboration is welcome

Autonomous Unmanned Vehicle (AUV) Workbench builds missions,

tracksModels air, surface, underwater, ground

robots• Currently work: remote operated vehicle

(ROV)• Open-source Java, XML, etc. with online

archivesAutonomous Vehicle Command Language

(AVCL): 3-level XML-based language for robot telemetry, mission orders, mission planning• Multiple converters written

• Tools for doing work! Mission displays, GIS, X3D graphics visualization, telemetry plots

6

Rehearsal

Prepare missions, either manually or automatically via other software tools

Test robot software’s ability to perform commands

Test again with physics “in the loop”• Hydrodynamics and control are critical,

difficult• Sonar, environmental modeling

Repeat until robust, with cautious respect• “Simulation is doomed to success” – G.

Bekey

7

Reality: real-time mission support

Monitor mission progressTask-level control using same mission

vocabularyVisualize and supervise operations

• caveat, again: work in progressIntegrate acoustic and RF

communicationsChat for distributed collaboration among

participants, both human and robotic

8

Record mission metadata for archives

Support operator keeping detailed notes, kept in context when conducting mission

Prompt for full details as appropriateArchive notes for later review and

followup

Future work• Automatic tests to confirm configuration,

control• Automate pre-underway checklists

9

Replay: post-mission supportAutomatic archiving of mission to server

• Being built into workbench – simplify user tasks

Integration and compression of all relevant data into single compressed XML file• Metadata for mission• Many pieces: ordered mission, commands,

telemetry, coefficients, contacts, etc. etc.• Autonomous Vehicle Control Language

(AVCL) is Ph.D. work by CDR Duane Davis

The Extended Rational Behavior Model

Features• Augments Existing

Vehicle Control Software• Translation to Vehicle-

Specific Data Format at Lowest Level

• AVCL Mission Construct Reliant

• Minimal Modification to Existing Vehicle Controller

Communications Model• Pull—Higher Levels

Query Lower Levels• Push—Higher Levels

Command Lower Levels• Push—Lower Levels

Provide Unrequested Information

Strategic Level

Tactical Level

Execution Level(existing vehicle

controller)

Declarative AVCL Mission High-Level

World Model

Mid-Level World Model

State and Sensor Data

Translator

Task-Level Behavior Script

Vehicle-Specific Com

mand

Status Q

ueryStatus Q

uery

Com

man

d St

atus

Script Status

Script Control

Agenda Control

Planner

Task-Level Behavior

Goal

Command FlowImplied Data VisibilityOptional Communications

Extended Rational Behavior Model Controller

AVCL track, planned and

simulated

AVCL planned mission track into KML

AVCL simulation track to KML

Extensible 3D (X3D) GraphicsISO Standard, developed by working-

group members of non-profit Web3D Consortium

Broad range of 3D graphics, multiple codebases

Patterned after HTML and Web Architecture• Evolved Virtual Reality Modeling Language

(VRML)• XML, ClassicVRML, compressed binary

encodingsArchival publishing of 3D models• Geometry, interaction, navigation, DIS

networking• Many many resources, exemplars, archives,

etc.

AUV WORKBENCHWORK IN PROGRESS

Slow but steady progress

AUV Workbench efforts in progress

AUV Workbench• Mission metadata and mission reports• Restructured threading for performance,

debugging• Testing AVCL support: AllCommandsUuv.xml

etc.• Joystick input: ROV or human-substitute

controlXj3D visualization improvements• Rendering support for 64-bit architectures• Group merging multiple builds into common

trunkNightly builds and releases

AllCommandsUuv.xmlList and test every AVCL command• Simple testing of workbench software• Confirms interface and validation• Not necessarily a runnable mission• Considering adding “dead reckoning” to 2D

plot• Mission unit tests for comprehensive

coverage likely to followAlso for other robotics platforms• AllCommandsUsv.xml AllCommandsUav.xml

AllCommandsUgv.xml

Joystick inputMotivation• Rehearse ROV missions, especially for high-

school student experiments and competitions

• Provide alternate control input when rehearsing complex UUV missions

• Make AUV Workbench suitable for classroom or poolside use

Software• Java jinput project handles different devices

and operating systems satisfactorily

Thoughts on autonomous ethicsEverything old is new again• Largely unexplored or unconsidered

territory?• Significant history of experience: mine

warfare?Assume success… then what?• Even if perfectly executable, proper robot

logic is not useful unless it is a directly extension of warfighter logic

• Rules of Engagement, Concepts of Operations, Doctrine, Tactics, etc. must be expressible in equivalent terms in order to be effective, usable

Multi-layer robot architectures

3 common layers defined in Rational Behavior Model (RBM) robot architecture• Strategic: planning, goal evaluation,

strategy• Tactical: navigation, operations, mission

conduct• Execution: low-level control of systems

Autonomous Vehicle Command Language (AVCL) mappable to many robot dialects

Possible fourth layer: Rules of Engagement?• provide ethical constraints on robot

strategies

AUTONOMOUS VEHICLE COMMAND LANGUAGE (AVCL)

XML vocabulary for robot mission specification, conduct and replay

Problem: Mission Incompatibility for Dissimilar Autonomous Vehicles

(AVs)Vehicle-specific data formats and mission

planning systems preclude effective coordination in multi-vehicle systems and hinder the design of such systems.

To date, the preponderance of multi-vehicle research has assumed inherent compatibility: homogeneous vehicle systems or explicitly programmed compatibility.

Proposed Solution: A Common AV Data Model

Common Data Format: (e.g., AVCL):

• Mission Specification (tasking)• Communications• Mission Results

Automated Conversions• AVCL to Vehicle-Specific• Vehicle-Specific to AVCL

Ultimate Goals• Facilitate interoperability

between dissimilar vehicles (including legacy)

• Support pre-, mid-, and post-mission data requirements

• Provide other vehicles and human operators intuitive and efficient data access

AUV 2

ASV 2

AAV 2

AAV 1 Suppor

t

Support

Support

Support

Support

Support

ASV 1

AUV 1

AVCL

AAV 1

Support

AUV 2

ASV 2

AAV 2

AUV 1

ASV 1

AVCL DescriptionAutonomous Vehicle Control Language

(AVCL) is a command and control language for autonomous unmanned vehicles, enabling common XML-based representations for mission scripts, agenda plans and post-mission recorded telemetry. Operators can utilize a single archivable and validatable format for robot tasking and results that is directly convertible to and from a wide variety of different robot command languages.

AVCL Integration in a Multi-Layer AV Control Architecture

Multi-Layer AV Control Architectures

• Hierarchical• Hybrid

Relationship to AVCL• Declarative Mission• Scripted Mission

Rational Behavior Model (RBM)• Three-Level Hybrid Architecture• Strategic Level: Ship CO• Tactical Level: Watch Officers• Execution Level: Watch Standers

RBM vs AVCL:Strategic Level: Declarative Mission

Tactical Level: Behavior Script

Execution Level: Individual Commands

AVCL and Related Technologies

Exemplar Data Model Definition: AVCL• Task-Level Behavior Set Determination• XML Schema Design• Correlated information:

• Mission Specification (goals, constraints, tasking)• Mission Results (telemetry)• Communications (message set)

Data Conversions• AVCL translation to Vehicle-Specific languages• Vehicle-Specific language translations to AVCL• AVCL Declarative output as AVCL Task-Level

commands• AVCL Task-Level status responses to AVCL

Declarative

Data Model Use to Facilitate AV Control• Integration into a Multi-Layer Architecture

Scripted Mission Specification

Task-Level Behaviors• Minimum Requirements

to Capture Arbitrary Tasking

• Behavior Activation and Termination Criteria

Scripting• Atomic Task-Level

Behaviors• Sequential Execution• Potential Parallelism

MakeSpeed

MakeDepth

Waypoint

MakeAltitude

Waypoint

ObtainGPS

SetPosition

Waypoint

Quit

Declarative Goal-Based Mission Specification

14 Predefined Goal Types

Finite State Machine (FSM)• States Represent

Individual Goals• Transitions Executed

upon Goal Success or Failure

Mission Start

Rendezvous with UUV-2 in

Area C

Search Area A

Search Area B Sample Environment in

Area A

Mission Complete

SucceedFail

SucceedFailSuccee

d or Fail

Succeed or Fail

AVCL TRANSLATIONS AND PARSER PRODUCTION

Theory and Practice

External C2 Systems

AVCL

Mission Goals and Constraints

Task Level Commands

Vehicle-Specific Languages

PlannerRules & Templates

XSLT and Vehicle-Specific XML

C2IEDM Business Objects

AVCL Translations OverviewAVCL to Vehicle-Specific Data

• XSLT (text)• Vehicle-Specific XML

(binary)Vehicle-Specific Data to AVCL

• Context Free Grammars (text)

• Vehicle-Specific XML (binary)

Artificial Intelligence• Planning and Search• Case-Based Reasoning• Bayesian Reasoning

Business Objects• Command and Control

Information Exchange Data Model (C2IEDM)

• AVCL Declarative Tasking

CFGs or Vehicle-Specific XML

AVCL to Text—Direct generation with XSLT

Exemplar Translations• AVCL to Phoenix UUV• AVCL to ARIES UUV• AVCL to Seahorse UUV• AVCL to REMUS UUV

XSLT Stylesheet

Vehicle-Specific Script

AVCL Script

Translation from AVCL to Text-Based Vehicle-Specific Formats

A Mutable Variable Pattern for XSLT

Why?• Inability to Update

Variable Values in XSLT while looping (immutable properties)

• Requirement to Maintain Behavior Parameters for Parallel Execution

How?• Use Template Parameters• Explicitly Controlled

Iteration

begin XSLT processing variable B = sequential list of task-level behaviors apply template for B1 with default parameters d1 to dn

end XSLT processing

begin template for task-level behavior Bi with parameters p1 to pn

for k = 1 to n variable vk

if Bi updates pk

vk = new_pk

else vk = pk

generate required output for Bi apply template for Bi+1 with parameters v1 to vn

end template

AVCL to Binary• Generate vehicle-specific XML with XSLT• Serialize to binary

Binary to AVCL• Read from binary to vehicle-specific XML• Generate AVCL with XSLT

Exemplar Implementation• AVCL to Joint Architecture for Unmanned Systems (JAUS)

Messages• JAUS Messages to AVCL

Vehicle-Specific Binary

Custom Serializer

AVCL Data

XSLT Stylesheet

Translation from AVCL to Binary Vehicle-Specific Formats and Vice Versa

Vehicle-Specific XML

AVCL Script

XSLT Stylesheet

Vehicle-Specific Binary

Custom Reader

Vehicle-Specific XML

Translating Vehicle-Specific Text Formats to AVCL

Parse as a Context Free Language (CFL)

• Chomsky Normal Form (CNF) Context Free Grammar (CFG)

• Cocke-Younger-Kasami (CYK) Parsing Algorithm

• Yields a Binary Parse TreeTranslate Parse Tree to AVCL

• Depth First Traversal• Template-Based Translation

of Individual Parse NodesCFG Serves the Same Role as an XML SchemaParser/Translator Serves the Same Role as an XSLT Stylesheet

Example Chomsky Normal Form Rules:Mission -> LaunchCmd + MissionMiddleMission -> LaunchCmd + MissionEndMissionMiddle -> WaypointCmd + MissionMdlMissionMiddle -> SurfaceCmd + MissionMdlMissionMiddle -> WaypointCmd + MissionEndMissionMiddle -> SurfaceCmd + MissionEndMissionEnd -> WaypointCmd + RendezvousCmdMissionEnd -> SurfaceCmd + RendezvousCmd

Example (Partial) Parse Tree

Mission

LaunchCmd MissionMdl

WaypointCmd MissionEnd

RendezvousCmdSurfaceCmd

Translation of Declarative Missions to Task-Level Behavior

ScriptsInter-Area Transit

• Best-First (A*) SearchGoal Accomplishment Planning

• Trivial Goal Types• MonitorTransmissions• Reposition

• Others• Require Area Coverage

(Search)• Decision-Tree Plan-Type

Selection• Parameterized Preplanned

Patterns for Regularly Shaped Operating Areas

• Planner-Generated (A*, Hill-Climbing, Iterative Improvement) for Irregularly Shaped Areas

TODO wish list for AVCL v2.1AVCL schema needs to be simplified• Remove vehicle-specific duplicate

definitions• Possibly streamline inner abstract types• Blocker: all changes reflected in JAXB API,

so XML changes provoke corresponding AUVW changes

Multi-vehicle message passing, coordination

Graduate improvements using MetaCommand• Joystick

Publish JAXB library as independent API

ReferencesAUV Workbench• https://savage.nps.edu/AuvWorkbench

AVCL page (with schema, DTD, documentation)• https://savage.nps.edu/Savage/AuvWorkbench/

AVCL/AVCL.html

Duane Davis Dissertation• Design, Implementation and Testing of a Com

mon Data Model Supporting Autonomous Vehicle Compatibility and Interoperability

Collaboration is welcomeNaval Postgraduate School (NPS)

is .mil, .edu university driven by research and education• Happy to offer our resources for your use• Happy to write proposals together to fund

workComing soon: 4 online courses as

distance-learning certificate• X3D graphics I and II, Simulation

Networking, Networked Virtual Environments (NVE)

Please tell us what you think. Thanks!

Don Brutzman, Ph.D.brutzman@nps.navy.mil

brutzman@nps.navy.smil.mil http://faculty.nps.edu/brutzman

Code USW/Br, Naval Postgraduate SchoolMonterey California 93943-5000 USA

1.831.656.2149 work1.831.402.4809 cell

Contact

Duane Davis, CDR USN, Ph.D.dtdavi1@nps.edu

Code CS/Dv, Naval Postgraduate SchoolMonterey California 93943-5000 USA

1.831.656.7980 work

Contact