Development of Autonomy for Unmanned Surface …terpconnect.umd.edu/~skgupta/USSV.pdfDevelopment of...

Development of Autonomy for

Unmanned Surface Vehicles

Satyandra K. Gupta, Bob Kavetsky, Steve Lubard,

Dana Nau, Eric Raboin, Brual C. Shah, Max

Schwartz, Petr Svec, and Atul Thakur

University of Maryland, College Park

Sponsor: Office of Naval Research

Challenges in Achieving USV Autonomy

• USSV will need to perform a large

number of tasks

– Different tasks require different

behaviors for achieving autonomy

– Missions may involve intelligent

adversaries

– Handling contingencies requires

additional behaviors

• The process of humans

developing the logic for autonomy

components and coding it

requires significant effort!

Unmanned Sea Surface Vehicle

(USSV)

Planning and Control Software for

Autonomous Operations

Task Planner

Behavior Selector

Trajectory Planner

Context

Dependent

Vehicle

Capability

Model

Behaviors

Nominal Planner

Refinement Planner

Actuator

Commands

Sensor Data

Maneuvers

USSV ONBOARD COMPUTING

Manually generating

these components

takes long time!

Exception

Exception

USSV

• Automatically generate behaviors and context dependent vehicle capability models for USV from a large number of simulations and demonstrations

– Simulations are getting faster and more accurate with faster computers

– Behaviors can be audited and edited to improve if distance < 20 then

rudder angle = 30;

throttle = 40;

...

...

USV finds the right

behavior

USV explores possible actions to

limit the operation of an intruder

in the virtual environment

Project Goal

Actuator

Commands

Maneuvers Behaviors

Automatically Generated Software Libraries

Learning from

Demonstration

Simulation in Virtual Environment

Synthesis by Evolutionary

Computation Reasoning

Context Dependent Vehicle

Capability Models

Computationally

Efficient

High-Fidelity

Simulations

Task Planner Behavior Selector Trajectory Planner

Planning Software

Overview of Approach

Feedback

Control

Layer

Behavior

Layer

Trajectory

Planning

Layer

Task Planning

Layer Mission

Motion Goal

Task Sub

Goals Queue

Task Planner

Behavior

Selection

Model-Predictive

Nominal

Trajectory Planner

Task Sub

Goals

Dispatcher Exceptio

n

Exceptio

n

Exceptio

n

Atomic

Maneuvers Maneuvers

Traversability

Map

S

E

N

S

O

R

S

S

T

A

T

E

E

S

T

I

M

A

T

O

R

Traversability

Model

Behaviors

ACTUATORS

Model-Predictive

Trajectory

Refinement

Planner

Behavior Executor

Planning

Heuristics

Motion Control

System Controller

Parameters

Trajectory

Tracking

Layer

Reference

Computing

Algorithm

Waypoint

Management

Control

Allocation

USV

Guidance System

Control System

Navigation

System

Waves, wind,

currents

Actuator commands

Control

forces

𝜏

1) Developed a virtual environment – Stereo visualization

– Human vs. human and human vs. autonomy

– Enables evaluation of behaviors

2) Developed physics based meta-models – Six DOF dynamics model

– Geometric model simplification techniques

to speed up computations by factor of 28

3) Developed trajectory planners and target

following capability – Safe, efficient, and dynamically feasible

trajectory planning and target following in high sea states

4) Synthesized USV blocking behavior

against a human-competitive intruder – Intruder with probabilistic actions

– Synthesized behavior was comparable to

a hand-coded behavior

5) Synthesized strategies and behaviors

for multi-USV team to protect asset from

threat of possible adversaries – USVs had to deal with uncertainty in recognition of

intruder boats

Summary of Accomplishments

Buffer zone

Danger zone

Friendly certified

boats

Target USV

Intruder

• Developed interactive rate

physics-based virtual simulation

environment for USVs

– Modeled interactions of waves and

the vehicle

– Real-time ocean rendering

– Stereo visualization

– Intuitive user interface

– Modeling of boats, terrain, and

shorelines

– Modular software design

• Can be used for measuring

performance of hand coded and

computer synthesized behavior

P. Svec, A. Thakur, D.K. Anand, S.K. Gupta, and M. Schwartz. A simulation based framework for discovering planning logic for

autonomous unmanned surface vehicles. ASME Engineering Systems Design and Analysis Conference, Istanbul, Turkey, July, 2010.

Accomplishment #1: Virtual Environment

1) A. Thakur, and S.K. Gupta, Real-time dynamics simulation of unmanned sea surface vehicle for virtual environments. Journal of

Computing and Information Science in Engineering, 11(3), 2011, 2-11.

2) A. Thakur, P. Svec, and S. K. Gupta. GPU based generation of state transition models using simulations for unmanned sea

surface vehicle trajectory planning, Robotics and Autonomous Systems, In review.

Accomplishment #2: Physics-Based Meta-Models

Boat

geometry Ocean

Model

Simplify based on

clustering

Simplify based on

temporal coherence

Parallelization on CPU

cores using OpenMP

Parallelize Monte Carlo

runs

Parallelize each run

across wet facets

Simplify based on

temporal coherence

CPU based simplification GPU based acceleration

Initial

State

• Developed high-fidelity simulation model and

corresponding physics based meta-model

– Six DOF dynamics model

– Geometric model simplification techniques to

speed up computations by factor of 42

– Combined with GPU computing for further

speed up

0

10

20

30

40

50

1 4 16 64 256

Sp

eed

up

facto

r o

ver

CP

U b

aseli

ne

Number of simultaneous runs GPU acceleration

CPU based simplification (τ=0.07, C=50 Thakur and Gupta 2011) GPU and temporal coherence based simplification τ=0.075

Accomplishment #3: Trajectory Planning

1. Manually designed

control actions

Au

Cu

Bu

'B

u

'C

u

Au

Cu

Bu

'B

u

'C

u

Au

Cu

Bu

'B

u

'C

u

2. Monte Carlo simulations of

control actions

3. Manually designed

probabilistic transitions

4. 3D lattice based trajectory

planning over space of (x, y, θ)

5. Localized search in space of

probabilistic transitions

Expected

collision

probability:

0.001 1.0

0.0 0.0

Collision

probability

0.1

0.01

0.1

0.8

0.1

Motion Commands Computed Control Set Variation in Final Orientation

• A state transition maps a given

state and action combination to

another state

• Motion commands are

implemented as position tracking

using PID controller

• Position uncertainty is captured by

running sufficient number of

simulation trials θ =2π

θ = 0

si

X Nominal

trajectory

obtained for

action u with

no ocean

disturbance

se

sj X X X

X

Ending state

for kth

simulation

run X XX

X X

XX

X X

X XX

X

X

XX X

X X X X

X

Accomplishment #3.1: Computing State Transition

Probability Using High-Fidelity Simulations

A. Thakur, P. Svec, and S. K. Gupta. GPU Based Generation of State Transition Models Using Simulations for Unmanned Sea

Surface Vehicle Trajectory Planning, Robotics and Autonomous Systems, In review.

Trajectory for sea state 3

(avg. wave height 0.5 to 1.25 m)

Trajectory for sea state 4

(avg. wave height 1.25 to 2.5 m)

Value iteration used for obtaining optimal feedback plan

A. Thakur, P. Svec, and S.K. Gupta. Generation of State Transition Map Using Unmanned Sea Surface Vehicle (USSV) Simulation,

ASME 2011 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference

(IDETC/CIE ‘11), 2011.

Accomplishment #3.2: Computing Trajectories For

Various Sea States

P. Svec, M. Schwartz, A. Thakur, and S.K. Gupta. Trajectory Planning with Look-Ahead for Unmanned Sea Surface Vehicles to

Handle Environmental Uncertainties, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '11), San

Francisco, September 25-30, 2011.

Accomplishment #3.3: Incorporating

Look-Ahead Into Trajectory Planning

• Standard stochastic dynamic programming

algorithms too slow to handle motion

uncertainty for nonholonomic vehicles

• A* or D* based heuristic search alone does

not handle motion uncertainty

• Developed a look-ahead based algorithm

for trajectory planning under motion

uncertainty for USVs

• Motion uncertainty is modeled

and explicitly used during the computation

of the trajectory

• Contingency plan is generated to efficiently

handle sudden large deviations from the

intended trajectory

• Developed follow behavior for tracking

moving target in complex environment

with obstacles

‒ Tracking influenced by USV dynamics,

motion characteristics of moving target,

and complexity of environment

‒ USV does not know trajectory of target

in advance

‒ USV predicts future pose of target based

on its motion characteristics and

configuration of obstacles

‒ USV keeps target within user-specified

distance range (𝑟𝑚𝑖𝑛, 𝑟𝑚𝑎𝑥)

‒ USV computes dynamically feasible

trajectory to reach target

‒ USV applies velocity and position

feedback control to reject disturbances

due to ocean waves

Accomplishment #3.4: Target Following with

Motion Prediction for USVs Operating in Cluttered Environments

Accomplishment #3.4: Target Following with

Motion Prediction for USVs Operating in Cluttered Environments (Cont.)

Buffer zone

Danger zone

Friendly certified

boats

Target USV

Intruder

• Case study: USV supposed to

block the advancement of an

intruder boat

─ Needs a reactive action selection

policy to compute motion goals

• Intruder exhibits a human-

competitive, deceptive

attacking behavior

─ USV cannot exploit regularity in

the intruder’s behavior due to its

probabilistic actions

Accomplishment #4: Action Selection Policy

Synthesis For Blocking Task

P. Svec and S.K. Gupta. Automated Synthesis of Action Selection Policies for Unmanned Vehicles Operating in Adverse

Environments. Autonomous Robots, Accepted for publication.

• Representative portions of

intruder’s attacking behavior

Target

Target


Synthesis For Blocking Task (Cont.)



Termination

criterion

met?

No

Yes

Find representative

set of states of

exception REPE

S,

Evolve policy

component to

handle REPE

S,

i

Optimize and

integrate into

the overall policy i

U

Evaluate policy U

States of

failure F

S

Representative

set of states of

exception REPE

S,

Policy

component

ASii:

Local

test cases

Default policy

Evolved

part

Hand-coded

part

D

Test cases T

Generate

local test

cases i

T

iT

USV policy

UUAS :

Return U

Intruder policy

IIAS :

World

Coordinate

Space

1) Detect failure state sF

2) Find

exception

state sE

by going

back in

time

Iterative State Space

Partitioning

3) Synthesize action Ai for

sE and its neighboring

states

sE





Automated

Policy Synthesis

Approach





Hand-

coded

average





Example of a run in which the USV managed to block the intruder for additional 45

seconds. The start position of the USV is marked as 1.1 while the start position of

the intruder is marked as 2.1.

• Synthesized strategies and

behaviors for multi-USV team to

protect asset from the threat of

possible adversaries

‒ USVs must deal with uncertainty about

which boats are adversaries

‒ Difficult task allocation due to multi-task

vehicles, multi-vehicle tasks, and time-

extended assignment

• Online simulation-based task

allocation

‒ Used predictive simulation for

task allocation evaluation

• Behavior learning through

parameter optimization

‒ Must learn behaviors for each

task

Danger

Zone

Asset

USV

2 Intruder

b2

P(b2 is intruder)=0.8


USV1

USV3

Boat b1 Intruder b3


• Team of 5 USSVs

• 6-8 passing boats

• 2-3 intruders,

Result of 1000

randomly generated

trials

• Predictive is 26%

better than baseline

Accomplishment #5: Generating Behaviors to

Support Multi-USSV Guarding

For more information please contact:

Satyandra K. Gupta ([email protected])

Development of Autonomy for Unmanned Surface …terpconnect.umd.edu/~skgupta/USSV.pdfDevelopment of...

Documents

Transcript of Development of Autonomy for Unmanned Surface …terpconnect.umd.edu/~skgupta/USSV.pdfDevelopment of...