1

Cooperative and Noncooperative Operations

of Swarms

Hoam Chung, David Shim, Mike Eklund, Shankar Sastry

University of California, Berkeley

www.swarms.org

SWARMS

2

Cooperative Operations of Swarms

Heterogeneous formation flight of MD500s and UH-60s

Various helicopter formations are now being used in many applications

Some level of automation during formation flight can reduce pilot stress and fatigue

Few research results on autonomous helicopter formation exist due to helicopter’s complicated dynamic properties, and technical difficulties

SWARMS

3

Mesh Controller Mesh Controller tasks:

Obtain the leader and 2 neighboring helicopters’ current positions

Compute mesh stable trajectories based on the acquired position information and send commands to the navigation computer

FlightComputer

MeshController

RS-232Wireless

Token Ring

Neighbor 1

Neighbor 2

Leader

On UAV

SWARMS

4

2001 Experiment

SWARMS

5

2001 Experiment

Animation by A. Pant and X. Xiao

Withoutleader

info

Withleader

info

SWARMS

6

Mesh Stable Controllers are OK, but…

The use of leader information improves the performance of the autonomous formation flight

For a heterogeneous mesh, an extension of mesh stability theory should be considered

“Mesh Stability” does not mean the “Safety”

It’s a starting point for autonomous formation flight

SWARMS

7

Model Predictive Control

Computes control inputs using real-time optimization

Shows better performance than non-predictive controls

Can consider various safety constraints in on-line manner Easily accommodates adaptive

disturbance rejection algorithms

SWARMS

8

Model Predictive Control

Compute control inputsminimizing gap errors

considering helicopterdynamics

at every sampling time

optimization can dealwith various constraints

Positions of neighboring

vehicles

Informationof formation

velocitiesdesired

gaps

Structure of MPC

Controlinputs

Weather conditions/Mission characteristics

SWARMS

9

Simulation Scenario 1

• 3DOF Point mass model• Homogeneous formation• Echelon right (45 deg. off lead)• Forward flight at 67.5 mi/h• Disturbance on 2nd helicopter• No safety constraints, no explicit disturbance rejection

t

n

Heli0

Heli1

Heli2

Heli3

* Formation from FM 1-112 Attack Helicopter Operations, Headquarters, Dept. of the Army

SWARMS

10

Animation

Animation generated by MATLAB

SWARMS

11

Simulation

Mesh stability is achieved without any explicit disturbance rejection algorithm

SWARMS

12

Bi-directional Information Flow

For safer autonomous formation, the communication between neighbors should be bi-directional

In case of mesh stability concept, it’s difficult to deal with bi-directional information

What will happen if directions of disturbancesare reversed?

Information flow

SWARMS

13

Bi-directional Information Flow

j

In case of MPC, simple redefinition of error signal can deal with bi-directional information flow

For example, we can redefine the error vector of jth helicopter so that - Keep the center between j-1 and j+1 in tangential direction - Keep the desired gap in normal direction

j-1

j+1

t

n

This flexibility of MPC allows various formations in 3D space with enhanced safety

SWARMS

14

Forming a Formation Adding vehicles one by one

1. Establish communication with vehicle A2. Acquire variables about existing formation

from vehicle A3. Compute a merging trajectory and track it4. Finish merging procedure if the gap error

is within a certain bound5. Engage formation controller

Merging procedures on the vehicle B:

A

B

SWARMS

15

Forming a Formation Adding vehicles group by group

1. Vehicle b establishes communication with vehicle a in A

2. Vehicle b acquires variables about leading formation from vehicle a

3. Compute merging trajectory

4. Propagate acquired variables and computed trajectory through B

5. Track the merging trajectory

6. Finish merging procedure

Merging procedures on the group B:

A B

+

a

b

a b

SWARMS

16

Terminating a Formation Terminating a formation one by one

1. Compute a trajectory to get more gap from the existing formation

2. Notify termination schedule to vehicle A3. Track the computed trajectory4. Send “Separation Completed” to vehicle A

and close the communication channel5. Disengage formation controller and give

control back to pilots

Termination procedures on the vehicle B:

A

B

SWARMS

17

Terminating a Formation Terminating a formation group by group

1. Compute a trajectory to get more gap from the leading formation

2. Propagate the computed plan to followers

3. Notify termination schedule to vehicle a

4. Track the computed trajectory

5. Send “Separation Completed” to vehicle a and close the communication channel between a and b

Termination procedures on b:

A B

a

ba b

SWARMS

18

Modifying a Formation in the Air

Modification of a formation MPC is basically a tracking controller By manipulating local formation variables(gap info), reconfiguration

of a formation without reorganization can be easily achieved

SWARMS

19

Simulation Scenario 3

• 3DOF Point mass model• Heterogeneous formation• 3D Vee formation (45 deg. off lead, 5m gaps in n, t, and z)• Forward flight at 67.5 mi/h, 5m(about 1.7 rotor radius) spacing• Disturbances on the leader and the last follower in right wing• No safety constraints, no explicit disturbance rejection

t

n

Mass Ratio

Heli0 100%

Heli1 200%

Heli2 300%

Heli3 100%

Heli4 300%

Heli5 100%

Heli6 200%

Heli0

Heli1

Heli2

Heli3

Heli6

Heli5

Heli4

SWARMS

20

Simulation of formation split and rejoin

SWARMS

21

Formation Rejoining

Consider a situation that a vehicle is approaching to the existing 3D Vee formation for joining

1

2

Safe region

SWARMS

22

Formation Rejoining

Objectives for a perfect formation rejoining A joining vehicle is positioned at predefined location in the formation When it finishes the procedure, its velocities and heading should be

close enough with those of the entire formation During the procedure, the joining vehicle should remain in a safe region

The motion of the future neighbor acts like a disturbance during the joining procedure For the vehicle in the formation, the first priority is maintaining the

formation Disturbances deteriorate ideal navigation conditions always exist

SWARMS

23

Formation Rejoining

The formation joining problem can be regarded as a differential gaming under input/state constraints

Following question should be answered: Does RHC scheme guarantee reachability under

disturbances? If so, how close is the reachable set rendered by RHC to

that by infinite-horizon problem?

SWARMS

24

Finite-horizon Differential Game

SWARMS

25

Finite-horizon Differential Game

The reachable set by the solution of FHODG problem is identical with that of a modified infinite-horizon problem

As becomes small, the reachable set of RHC approaches to that of infinite-horizon solution with

SWARMS

26

Finite-horizon Differential Game

This lemma plays important role in designing a receding horizon controller satisfying the condition

SWARMS

27

Finite-horizon Differential Game

The reachable set can be enlarged by introducing longer prediction horizon

These theorems and lemma tell us that, if the FHODG is feasible with some prediction length L, then it guarantees a successful formation rejoining

SWARMS

28

Works in Progress

A RHC scheme will be designed for 3D nonlinear kinematics plus linear dynamics model

Various numerical methods are now being investigated Continuation method – Ravio et al. Piecewise linear approximation and SQP – Fabien Lagrange multipliers method – Sutton and Bitmead

For reducing computational burdens, the performance of open-loop and Stackelberg solutions under RHC scheme will be evaluated

The algorithm will be implemented and tested on BEAR hardware-in-the-loop systems

SWARMS

29

20

40

60

80

-20

0

20

40

25303540

Collision pt:(50,0,33)ft

Collision Avoidance using MPC

20

40

60

80

-20

0

20

40

25303540

•Five helicopters are given destination points.•The shortest (optimal) trajectory will lead to a collision.

•Each vehicle can detect other vehicles position only within the sensing/communication region.

•Can each vehicle fly safely and optimally?

Unsafe Desired Trajectory Resolved by NMPTC with Collision Avoidance

SWARMS

30

Collision Avoidance using MPC

•Two UAVs are intentionally set on a head-on crash course

• Model-predictive control-based trajectory planner computes safe trajectories with sufficient clearance in real time

• Each vehicle’s current coordinate is used for MPC at each computation

May 2003

SWARMS

31

Collision Avoidance using MPC

SWARMS

32

Obstacle Avoidance System

• Dynamic path planning: real-time path generation using model predictive control

• Sensing: onboard 3D laser scanner or preprogrammed obstacle maps

• Experiment system: Berkeley UAV architecture implemented on Yamaha industrial helicopter platform with 3D laser scanner

min/O BX

SWARMS

33

Urban Flight Experiment10’ X 10’ Easy-up Canopy

• 6 canopies to simulate urban environment• Secured by stakes at four corners• Resistant to wind gust of rotor downwash• Sufficient distances each other for helicopters to fly through

Original path

Adjusted path by MPC

Vehicle Launching Pt.

Ground Station

Obstacles

Richmond Field Station, UC Berkeley, Richmond, California

SWARMS

34

Urban Navigation Experiment

35

Non Cooperative Actions of Swarms:

David Shim, Jongho Lee, Mike Eklund, Jonathan Sprinkle, Shankar Sastry

SWARMS

36

Aerial Pursuit-Evasion in MPC framework

Pursuer wants to position itself in a good position to “shoot down” the evader, e.g., follow the target’s tail and align its heading with the relative position vector, XE-XP

Evader wants to shake off the pursuer, e.g., get out of the hotspot

Pursuer and evader avoid colliding into each other within a closed 3-D space

State variables such as roll and pitch angles are constrained

SWARMS

37

Pursuer and Evader in a closed 3-D space with the additional cost

Aerial Pursuit-Evasion in MPC framework

Cost function also includes collision avoidance between aircraft and other obstacles including terrain

Illustration by Mike Eklund and Jonathan Sprinkle

SWARMS

38

ANIMATION 2004

SWARMS

39

Multiplayer PEGs: Proposed Solution

A close analogy is football: Multi player Initial (global) strategies well

defined Limited (local) coordination

after the snap

What can we learn? How can we apply this? How far does the

analogy go?

Back

SWARMS

40

Multiplayer PEGs Preseason (Off-line precomputed strategy)

Play book: Evaluate strategies and configurations that will maximize chance of success

based on best estimate of other team’s tactics Practice and preseason games:

Test playbook and find problems Game time (On-line adaptive strategy)

Choose play based on best knowledge and experience Line up (in best detection configuration, not necessarily static)

Execute the play Active and reactive actions (respond to detected evader) Local communication Adapt to evolving behavior

Learn from experience, repeat as necessary (Learning by Doing)

Back