Bilateral Teleoperation of Multiple Cooperative Robots over

Delayed Communication Network: Application

Dongjun LeeMark W. Spong

Oscar Martinez-Palafoxd-lee@control.csl.uiuc.edu, {mspong,pomartin}@uiuc.edu

Research partially supported by the Office of Naval Research (N00014-02-1-0011 and N00014-05-1-0186), the National Science Foundation (IIS 02-33314 and CCR 02-09202), and the College of

Engineering at the University of Illinois.

Outline

1. Review of the Proposed Control Framework

2. Simulation Results

3. Semi-Experimental Results

4. Conclusions

Bilateral Teleoperation of Cooperative Multi-Robots

Combine advantages of - bilateral teleoperation: human intervention in uncertain environments - multi-robot cooperation: mechanical strength/dexterity & robustness/safety

- applications: remote construction/maintenance of space/under-water/civil structures in possibly hazardous environments

Semi-Autonomous Teleoperation

- Passive Decomposition [Lee&Li, CDC03] decomposes slave dynamics into

decoupled shape (formation shape) and locked (overall group motion) systems

- Local grasping control of decoupled shape system: secure/tight grasping regardless of human command via delayed comm. Channel

- Bilateral teleoperation of locked system: by operating the master robot of manageably small DOF, human can tele-control the behavior of the grasped object over the delayed comm. channel while perceiving external forces

internal formation shape (cooperative grasping)

Shape System

behavior of overall group(and grasped object)

Locked System

Coupling:dropping object!!!

Passive

decoupling

System Modelling and Grasping Shape FunctionDynamics

of a single

master(m-DOF) Dynamics of multipleslave robots

(n1+n2+…+nN-DOF)

n-DOF product system(n=n1+n2+…+nN-dimensional)

Stack-up

inertia Coriolis control human force

velocity

grasping shape function describes internal group

formation shape

Grasping Shape Function: Rn→Rn-m

desired (constant)grasping shape

master’s DOF

locked system

shape system

Passive Decomposition and Local Grasping ControlDecomposed Slave

Dynamicspassive

decoupling

Local Grasping Control

FF cancellation of internal force: although dynamics is decoupled, other effects (e.g. object’s inertia) can still perturb the shape system through internal force FE

desired grasping shape

Locked system: abstracts overall behavior of multiple slave robots and grasped object

Shape system: describes internal group formation of slave robots (i.e. cooperative grasping)

Scattering-Based Teleoperation of Locked System

controlhuman/combined external forces

Dynamics of Master Robot

and Slave Locked System(both are m-

DOF)

Shape system(locally controlled)

Locked System

Scattering-Based Teleoperation of Locked system: - humans can tele-control the behavior of the grasped object

over delayed comm. channel while perceiving external forces acting on the

object and slaves - asymptotic position coordination/static force reflection

Outline

1. Review of the Proposed Control Framework

2. Simulation Results

3. Semi-Experimental Results

4. Conclusions

Simulation Settings

- grasping shape function is defined s.t. three slaves form an equilateral triangle

(w/ side length L) whose rotation is specified by the heading of agent 2

- human operator can tele-control the position and rotation of the triangle

by operating 3-DOF master robot (translation and yaw)- 10% identification errors for inertias of robots (nominal:

m=1kg, I=1kgm2)

Delay 0.5s

Delay 0.5s

3-DOF Master

(x,y)-translationyaw rotation

Three 3-DOF Slave Robots

deformableobject

(no friction)

agent1

agent2agent3

Simulation: Importance of Decoupling

- no grasped object (just motion coordination) w/ PD-based grasping control

- without decoupling control, grasping shape (i.e. shape system) is perturbed

by human command and overall group behavior - slight grasping shape distortion w/ decoupling is due to

inertial uncertainty

Without Passive Decoupling Control With Passive Decoupling Control

Simulation: Heavy Object Fixtureless Manipulation

- even if dynamics is decoupled, inertial effect of object (w/ frictionless contact) perturbs cooperative grasping through the internal force FE

- this perturbation can be cancelled out by feedforward cancellation of the internal

force FE (or also by large enough PD-gains)

With Feedforward Cancellation of Internal Force

Without Feedforward Cancellation of Internal Force

Heavy Object Manipulation: Contact/Human Force

- human can perceive the total inertias of the grasped object and the slave robots

- human can also perceive sensation of grasping loss - better load-balancing is achieved w/ FF-cancellation of the

internal force FE, as grasping shape becomes more rigid

good load balance due to grasping

rigidity

due to grasping shape

deformation

Simulation: Force Reflection

- external forcing (x-direction) on the grasped object is faithfully reflected to the

human operator (i.e. haptic feedback)- load balancing among slaves is degraded as the grasped

object is deformed in the rigidly-maintained grasping shape

Three 3-DOF Slave Robots

deformableobject

agent1

agent2agent3

external force

due to object’sdeformation

humanforce

Outline

1. Review of the Proposed Control Framework

2. Simulation Results

3. Semi-Experimental Results

4. Conclusions

Semi-Experiment Setting

- three slave robots: 2-DOF point mass dynamics (only x,y translations)

- Phantom Desktop is used as master with its workspace constrained on (x,y)-plane

- Grasping shape function:

: specifies rotation and shape of the triangle formed by the three slaves

1 2 41 2 3

2 3

( , , )E

q qq q q q

q q

Delay 0.5s

Delay 0.5s

2-DOF Master

PHANToM Desktop:constrained on plane (i.e. (x,y)-translation)

Three 2-DOF Slave Robots

agent1

agent2

agent3

deformableobject

external force

Semi-Experiment: Deformable Object Manipulation

- x-directional motion (full-range) w/ fixtureless grasping- grasping security is preserved regardless of human command- human can perceive the combined inertia of slaves and

grasped object- increase of some slaves' contact force due to

inertia/deformation of object

secure/precise grasping w/ FF-term

due to object deformation

human perceivesinertias of object/slaves

Semi-Experiment: Obstacle Perception

- external force (x-direction) on the grasped object center - force generated by the PI-action in the local impedance

controls- object’s deformation again leads in unbalanced load sharing

among slaves

human perceivesexternal force

Secure/precise grasping w/ FF-term

due to object deformation

ConclusionsWe propose a control framework for bilateral

teleoperation of multiple cooperative robots over delayed master-slave comm.

channel: - passive decomposition: the decoupled shape

(cooperative grasping) and locked (behavior of the grasped object)

systems - local grasping control for the shape system: high

precision cooperative grasping regardless of human

command/comm. delays - scattering-based bilateral teleoperation of the locked

system: human can tele-control behavior of the

cooperatively grasped object by operating a small-DOF of the master

robot, while perceiving combined force on the slaves and the

grasped object over the delayed comm. channel - enforce energetic passivity: interaction safety and

stability

- Semi-experiment and simulation results are presented and

validate efficacy of the proposed control framework

Possible impacts on emerging or traditional applications: - remote construction/maintenance of space/under-

water/civil structures in hostile/hazardous environments