Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

73
 University of Tennessee, Knoxville Trace: Tennessee Research and Creative Exchange Masters eses Graduate School 8-2011 Ha ptic T ele-operation of W heeled Mobile Robot and Unmanned Aerial Vehicle over the Internet Zhiyuan Zuo [email protected] is esis is brought to you for free and open access by the Graduate School at Trace: T ennessee Research and Creative Exchange. It has been accepted for inclusion in Masters eses by an authorized administrator of Trace: Tennessee Research and Creative Exchange. For more information, please contact [email protected] . Recommended Citation Zuo, Zhiyuan, "Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internet. " Master's esis, University of Tennessee, 2011. hp://trace.tennessee.edu/utk_gradthes/1044

description

fdsfsad dfsfdsa

Transcript of Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

Page 1: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 173

University of Tennessee Knoxville

Trace Tennessee Research and CreativeExchange

Masters eses Graduate School

8-2011

Haptic Tele-operation of Wheeled Mobile Robotand Unmanned Aerial Vehicle over the Internet

Zhiyuan Zuozzuo1utkedu

is esis is brought to you for free and open access by the Graduate School at Trace Tennessee Research and Creative Exchange It has been

accepted for inclusion in Masters eses by an authorized administrator of Trace Tennessee Research and Creative Exchange For more information

please contact traceutkedu

Recommended CitationZuo Zhiyuan Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internet Masters esisUniversity of Tennessee 2011hptracetennesseeeduutk_gradthes1044

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 273

To the Graduate Council

I am submiing herewith a thesis wrien by Zhiyuan Zuo entitled Haptic Tele-operation of WheeledMobile Robot and Unmanned Aerial Vehicle over the Internet I have examined the nal electronic copy of this thesis for form and content and recommend that it be accepted in partial fulllment of the

requirements for the degree of Master of Science with a major in Mechanical EngineeringDongjun Lee Major Professor

We have read this thesis and recommend its acceptance

Xiaopeng Zhao William R Hamel

Accepted for the CouncilCarolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on le with ocial student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 373

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final paper copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 473

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final electronic copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on file with official student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 573

Haptic Tele-operation of Wheeled

Mobile Robot and Unmanned

Aerial Vehicle over the Internet

A Thesis Presented for

The Master of Science

Degree

The University of Tennessee Knoxville

Zhiyuan Zuo

August 2011

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 2: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 273

To the Graduate Council

I am submiing herewith a thesis wrien by Zhiyuan Zuo entitled Haptic Tele-operation of WheeledMobile Robot and Unmanned Aerial Vehicle over the Internet I have examined the nal electronic copy of this thesis for form and content and recommend that it be accepted in partial fulllment of the

requirements for the degree of Master of Science with a major in Mechanical EngineeringDongjun Lee Major Professor

We have read this thesis and recommend its acceptance

Xiaopeng Zhao William R Hamel

Accepted for the CouncilCarolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on le with ocial student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 373

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final paper copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 473

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final electronic copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on file with official student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 573

Haptic Tele-operation of Wheeled

Mobile Robot and Unmanned

Aerial Vehicle over the Internet

A Thesis Presented for

The Master of Science

Degree

The University of Tennessee Knoxville

Zhiyuan Zuo

August 2011

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 3: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 373

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final paper copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 473

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final electronic copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on file with official student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 573

Haptic Tele-operation of Wheeled

Mobile Robot and Unmanned

Aerial Vehicle over the Internet

A Thesis Presented for

The Master of Science

Degree

The University of Tennessee Knoxville

Zhiyuan Zuo

August 2011

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 4: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 473

To the Graduate Council

I am submitting herewith a thesis written by Zhiyuan Zuo entitled ldquoHaptic Tele-

operation of Wheeled Mobile Robot and Unmanned Aerial Vehicle over the Internetrdquo

I have examined the final electronic copy of this thesis for form and content and

recommend that it be accepted in partial fulfillment of the requirements for the degree

of Master of Science with a major in Mechanical Engineering

Dongjun Lee Major Professor

We have read this thesisand recommend its acceptance

Xiaopeng Zhao

William R Hamel

Accepted for the Council

Carolyn R Hodges

Vice Provost and Dean of the Graduate School

(Original signatures are on file with official student records)

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 573

Haptic Tele-operation of Wheeled

Mobile Robot and Unmanned

Aerial Vehicle over the Internet

A Thesis Presented for

The Master of Science

Degree

The University of Tennessee Knoxville

Zhiyuan Zuo

August 2011

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 5: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 573

Haptic Tele-operation of Wheeled

Mobile Robot and Unmanned

Aerial Vehicle over the Internet

A Thesis Presented for

The Master of Science

Degree

The University of Tennessee Knoxville

Zhiyuan Zuo

August 2011

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 6: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 673

c⃝ by Zhiyuan Zuo 2011

All Rights Reserved

ii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 7: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 773

To my dear family

iii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 8: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 873

Acknowledgements

First I would like to thank my adviser Dongjun Lee who offers me the opportunity

to work on these challenging and interesting projects who guides me in the new area

of study and who serves as a wonderful example of researcher

I would also thank Dr Hamel and Dr Zhao for being my committee members andproviding valuable advice in my research and thesis writing Also thank the faculties

in EngineeringMath School who taught me those interesting and useful theories

Thank my friends in the my research lab INRoL Ke Daye Tian and Robert who

helped me a lot in working and study and who shared with me laugh and knowledge

Also thank my family who are watching and thinking of me all the time

iv

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 9: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 973

Abstract

Teleoperation of groundaerial vehicle extends operatorrsquos ability (eg expertise

strength mobility) into the remote environment and haptic feedback enhances the

human operatorrsquos perception of the slave environment In my thesis two cases

are studied wheeled mobile robot (MWR) haptic tele-driving over the Internetand unmanned aerial vehicle (UAV) haptic teleoperation over the Internet We

propose novel control frameworks for both dynamic WMR and kinematic WMR in

various tele-driving modes and for a ldquomixedrdquo UAV with translational dynamics and

attitude kinematics The recently proposed passive set-position modulation (PSPM)

framework is extended to guarantee the passivity andor stability of the closed-loop

system with time-varyingpacket-loss in the communication and proved performance

in steady state is shown by theoretical measurements For UAV teleoperationwe also derive a backstepping trajectory tracking control with robustness analysis

Experimental results for dynamickinematic WMR and an indoor quadrotor-type

UAV are presented to show the efficacy of the proposed control framework

v

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 10: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1073

Contents

List of Figures viii

1 Introduction 1

2 Background 8

21 Passive Set-Position Modulation 8

3 Wheeled mobile robot tele-driving over the internet 12

31 Problem Setting of WMR Tele-Driving 12

32 Dynamic WMR Tele-Driving Control 14

321 (q 1 ν )(q 2 φ) Tele-Driving 14

322 (q 1 ν )(q 2 φ) Tele-Driving 1933 Kinematic WMR Tele-Driving Control 21

331 (q 1 ν )(q 2 φ) Tele-Driving 21

332 (q 1 ν )(q 2 φ) Tele-Driving 25

4 Quad-rotor type UAV tele-operation over the Internet 26

41 Problem Setting of UAV teleoperation 26

42 Backstepping trajectory tracking control of UAVs 27

421 Controller Design 27

422 Robustness Analysis 30

43 Application to UAV teleoperation over the Internet 33

431 Virtual Point and Virtual Environment 33

vi

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 11: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1173

432 Control Design for Haptic Device 37

5 Experiment 40

51 WMR haptic tele-driving Experiment 40

52 UAV haptic tele-operation 47

6 Conclusions 51

Bibliography 52

Vita 59

vii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 12: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1273

List of Figures

11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1 to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 4

12 UAV teleoperation The velocity of the virtual point ˙ p is controlled bythe configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 5

21 Energetics of the passive set-point modulation 10

31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car

driving metaphor we use q 1

to control WMRrsquos forward motion (ν )

and q 2 WMRrsquos turning motion (φ or φ) 13

41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion

partϕT opartp

from the virtual obstacle The UAV follows the virtual point

through a backstepping controller 33

51 Unstable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim

035sec

randomly varying delay without using PSPM 41

52 Unstable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay without using PSPM 41

viii

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 13: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1373

53 Stable case dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

54 Stable case kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025sim035sec

randomly varying delay 42

55 Travel around two obstacles in an rsquo8rsquo figure shape 43

56 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2513ms for the

haptic device and 2963ms for the WMR 44

57 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 5077ms for the haptic

device and 9246ms for the WMR 44

58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact 45

59 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2549ms for the

haptic device and 3032ms for the WMR 45

510 Dynamic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly vary-

ing delay Average packet-to- packet interval 4931ms for the haptic

device and 8678ms for the WMR 45

511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)

running into a corner (around 9 sec) q2 is kept but the WMR steers

clear of the obstacle (during 9-14 sec) 46

512 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2472ms for the

haptic device and 2895ms for the WMR 47

513 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5044ms for the

haptic device and 8643ms for the WMR 47

ix

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 14: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1473

514 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly

varying delay Average packet-to- packet interval 2360ms for the

haptic device and 1939ms for the WMR 48

515 Kinematic WMR (q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly

varying delay Average packet-to- packet interval 5114ms for the

haptic device and 1838ms for the WMR 48

516 Context diagram for the UAV haptic teleoperation program 49

517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape The

trajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus05 cos(02πt) 50

518 UAV haptic teleoperation over the Internet with randomly varying

delay 025sec minus 035sec 50

x

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 15: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1573

Chapter 1

Introduction

Vehicle teleoperation finds its application in various cases planet exploration(Schenker et al (2003)) landscape survey (Valavanis (2007)) transportation and

rescue (Nourbakhsh et al (2005)) material handling (Peshkin and Colgate (1999)) or

deep sea exploration (Baiden and Bissiri (2007)) By tele-driving aerialgroundunder

water vehicles humanrsquos skill strength and mobility are extended to the remote

environment Besides haptic teleoperation enables human to operate by haptic

feeling This enhances environment awareness and improves efficiency especially

when the vehicle is in clustereddark environment (Lim et al (2003) Cho et al(2010) Lamb (1895)) Haptic feedback become more important if the communication

bandwidth is restricted or when the task requires awareness of certain mechanical

aspect (eg pushing force against the object inertia of the WMR)

Two cases are studied in my research wheeled mobile robot (WMR) tele-

driving (Zuo and Lee (2010)) and quad-rotor type UAV(unmanned aerial vehicle)

teleoperation through haptic device over the Internet (Zuo and Lee (2011))

For WMR tele-driving we aim to achieve car-driving metaphor that is we useone-DOF (degree of freedom) as the gas-pedal to control its linear velocity while the

other-DOF as the wheel to control its heading angle or angular rate Meanwhile

we want to offer useful haptic feedback indicating the motion of the WMR (eg

1

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 16: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1673

velocity heading angle or angular rate) and environment force (eg repulsion from

obstacles perceived through sensors) For UAV teleoperation we want to control its

translational velocity in the space by 3-DOFs of the haptic device and to perceive

velocity and environment force at the same time In both cases we need to cope

with the problem of time-varying delay and packet-loss in communication so that

the systems can work properly over the Internet

It is known that time delay can easily induce bilateral tele-operators to be

unstable and this problem is extensively studied since 1980s Among these

approaches passivity based methods like scattering based method (Anderson and

Spong (1989a)) or wave based method (Niemeyer and Slotine (1991)) have more

advantage when dealing with the uncertainties in operatorenvironment compared to

Leung et al (1995)rsquos micro-synthesis or Lawrence (1993)rsquos performance-oriented method

One the other hand communication through Internet brings about more challenges

like the instability caused by varying delay and packet-loss ScatteringWave based

method is modified to deal with varying delay (Chopra et al (2003) Niemeyer and

Slotine (1998) or packet loss (Berestesky et al (2004a) Hirche and Buss (2004) and

some new passivity based methods appeared eg PD-like control (Lee and Spong

(2006a) Nuno et al (2009a)) Passity-ObserverPassivity-Controller(Hannaford and

Ryu (2002)) or Passive-Set-Position-Modulation (Lee and Huang (2009b))

When it comes to tele-operating the mobile robot more difficulties appear

Instead of positon-position coupling positon-velocity coupling is required since the

workspace of the master device is bounded while the that of the slave robot is

unbounded The scatteringwave based method where mechanical power is taken

as the power supple is not suitable to solve this problem For this Salcudean

et al (2000) proposed the rate control yet lacked robust analysis Lee et al (2006a)

proposed a r-passivity method to achieve stable teleoperation

Meanwhile some works have been done for vehicle teleoperation For dynamic

WMRs a passivity-based control design is proposed by Lee et al (2006b) yet only

for the (q 1 ν )(q 2 φ) mode with constant delay assumption For kinematic haptic

2

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 17: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1773

tele-driving some methods are given by Lee et al (2002) and Lim et al (2003) yet

the effect of force reflection on the stability is not considered Another passivity-

based method is proposed by Diolaiti and Melchiorri (2002) by introducing a virtual

mass on the slave side yet the effect of communication delay between the virtual

mass and the haptic device on the passivity is not analyzed Communication delay

and its associated stability problems are considered by Slawinski et al (2007) and

Elhajj et al (2001) however the result by Slawinski et al (2007) involves only vision

feedback but no haptic feedback and the scheme by Elhajj et al (2001) divides the

continuous operation into events which may not suitable for continuous-time stability

analysis and its performance may deteriorate when delay is not small (around 300ms)

Besides all of these works except that of Lee et al (2006b) consider only the

kinematic WMRs thus cannot address (often important) mechanical aspects (eg

force between the environment inertia of WMR etc)

UAV teleoperation also becomes an popular topic recently Lam et al (2009)

propose an artificial field method suitable for UAV teleoperation yet the delay in

the communication is not considered Rodrıguez-Seda et al (2010) consider the

stability problem using PD-based control (Lee and Spong (2006a)) yet only for

position-position coordination with constant time delay Stramigioli et al (2010)

introduce a virtual vehicle controlled by haptice device and coupled with the actual

vehicle stability of the master and virtual vehicle is guaranteed by using Hamiltonian

method and a controller to couple virtualactual vehicle is proposed based on the

assumption that attitude dynamic is much faster than translational dynamic The

robustness of the controller for coupling is not clear Blthoff (2010) deal with the

stability problem using passive-set-position-modulation (PSPM) method and achieve

master-passivityVPs-stability

In the thesis we design frameworks for WMR tele-driving and quad-rotor type

UAV teleoperation over the Internet Stability is guaranteed and performance is

measured for the teleoperation system with varying-delaypacket-lossdata-swapping

in the communication

3

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 18: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1873

Figure 11 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

For WMR tele-driving we also adopt car driving metaphor Diolaiti and Melchiorri

(2002) Lee et al (2002 2006b) as illustrated in Fig 31 we use one-DOF (degree-

of-freedom) of the master device (eg q 1) to control the WMRrsquos forward velocity ν

(ie (q 1 ν ) tele-driving) while another-DOF (eg q 2) the WMRrsquos heading angle φ or

its rate φ (ie (q 2 φ) or (q 2 φ) tele-driving) By this car driving metaphor operators

can tele-drive the WMR as if they are driving a car with q 1 and q 2 being used as the

gas pedal and the steering wheel respectively

For the UAV teleoperation see Fig 41 we use 3-DOF of the haptic device to

control the velocity of the UAV A virtual point is introduced and simulated in the

virtual environment The virtual point velocity ˙ p is controlled by the configuration of

the hapitc device q and affected by the repulsion partϕT opartp

from the virtual obstacle Then

the actual UAV follows the virtual point through a backstepping controller The UAV

is modeled by a combination of translational dynamics and attitude kinematics on

SE (3) with the thrust λ (along one body-fixed frame - eg e3) and the two angular

rates ω1 ω2 (along the remaining two body-fixed axes - eg e1 e2) as the control

input This model can be used for many commercially available UAVs including

our laboratory systems Asctec Hummingbird only allow us to control their thrust

force and angular rates not their angular torques This is because usually they are

4

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 19: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 1973

Figure 12 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

shipped with some low-level attitude control module already in place Even if our

main focus is on such ldquomixedrdquo UAVs we also believe our results as proposed here

would also be applicable to many UAVs accepting the angular torques as their inputs

by achieving these desired angular rates which is very often possible since most of

UAVsrsquo attitude dynamics is fully actuated

In contrast to the case in conventional teleoperation where the workspace for

master and slave robot are bounded here we want to achieve position-to-velocity

coupling (eg (q 2 ν ) for WMR or (q x) for UAV) Thus the conventional approach

dealing with position-to-position coordination over Internet like Berestesky et al

(2004b) Nuno et al (2009b) cannot be readily used To solve this problem we utilize

the recently proposed passive set-position modulation (PSPM) framework (Lee and

Huang (2008) Lee (2008) Lee and Huang (2009ab) which can modulate set-position

signal received from the Internet within the tele-driving control-loop to enforce

(closed-loophybrid) passivity even if this set-position signal undergoes varying-delay

and packet-loss Due to the flexibility of PSPM (Lee and Huang (2009b)) we can

encode other information like the value of WMR velocity or virtual environment force

into the set-position signal (eg pν (t) in Fig 31 or y(t) in Fig 41) and achieve

position-to-velocity coordination and force reflection Also compared to other ldquotime-

invariantrdquo techniques for delayed-teleoperation (eg Anderson and Spong (1989b)

5

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 20: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2073

Niemeyer and Slotine (2004) Lee and Spong (2006b)) PSPM selectively triggers the

passifying action and substantially improves the performance

More specifically we first extend the framework proposed in Lee (2008) for the

(q 1 ν )(q 2 φ) tele-driving of the dynamic WMR by incorporating the scaling of

the master-slave power-shuffling This scaled power-shuffling was motivated by our

observation that the human energy shuffled via the PSPM to drive the WMR was

very often all dissipated by the WMRrsquos large dissipation (eg gearbox tireground

interaction etc) This power-shuffling scaling enables us to virtually scale up the

human power thereby addressing such high dissipation We also propose the PSPM

control frameworks for the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR and

for the (q 1 ν )(q 2 φ) and (q 1 ν )(q 2 φ) tele-driving of the kinematic WMRs The

flexibility of the PSPM allows us to achieve teleoperation of the dynamickinematic

WMR and UAV while retaining peculiarity of each tele-driving mode (eg usual

two-port passivity for dynamic WMR or a combination of passivitystability for

UAV)

As for the quadrotor-type UAV numerous control laws available (eg Frazzoli

et al (2000) Mahony and Hamel (2004) Castillo et al (2004) Bouabdallah and

Siegwart (2005) Aguiar and Hespanha (2007) Hua et al (2009)) and some of them

utilize backstepping control on SE (3) as we do in this paper (eg Frazzoli et al

(2000) Mahony and Hamel (2004) Aguiar and Hespanha (2007)) Yet derived for the

UAVs with the translation and attitude dynamics these backstepping control laws are

not directly applicable to and also unnecessarily complicated for such ldquomixedrdquo UAVs

with attitude kinematics (and control inputs λ ω1 ω2) Compared with others the

control law proposed in this thesis possess two advantages 1) it is flexible in the sense

that many control laws designedverified point mass dynamics can be incorporated

into our backstepping control framework 2) it is transparent in the sense that its

gain tuning can be done much more intuitively since all of its gain parameters can be

interpreted either as parameters of standard second-order dynamics (eg damping or

spring) or convergence factor of first-order dynamics For the real implementation

6

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 21: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2173

we also provide robustness analysis for our backstepping control law against mass (or

thrust) parameter estimation error and Cartesian disturbance (eg wind gust) and

show that as long as they are bounded the trajectory tracking is still ultimately

bounded Khalil (1996)

The rest of the thesis is organized as follows Chapter 2 presents some preliminary

materials about PSPM Chapter 3 gives the controller design for WMR haptic tele-

driving and Chapter 4 for UAV haptic teleoperation The experimental result is

shown in Chapter 5

7

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 22: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2273

Chapter 2

Background

21 Passive Set-Position Modulation

Consider the following second order robotic system

M (x)x + C (x x)x = τ + f (21)

where M (x) C (x x) isin realntimesn are the inertia and Coriolis matrix with xτf isin realn

being the configuration control and humanenvironment force respectively Suppose

we aim to coordinate x(t) with a sequence of discrete signal y(k) isin realn via a local

spring coupling with damping injection that is for t isin [tk tk+1)

τ (t) = minusBx(t) minus K (x(t) minus y(k)) (22)

The main problem of this simple coupling is that due to the switches of y(k) energy

in the spring K can jump accumulate and eventually make this coupling control

unstable

PSPM can passify these spring jumps by watching the energy in the system and

selectively modulating the set-position signal from y(k) to y(k) To describe PSPM in

Algo1 let us denote the estimate of damping dissipation as Dmin(k) with Dmin(k) le

8

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 23: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2373

Algorithm 1 Passive Set-Position Modulation

1 y(0) lArr x(0) E (0) lArr E k lArr 02 repeat

3 if data (y ∆E y) is received then

4 k lArr k + 15

y(k) lArr y ∆E y(k) lArr ∆E y6 retrieve x(tk) xmax

i (k minus 1) xmini (k minus 1)

7 find y(k) by solving

miny(k)

||y(k) minus y(k)|| (23)

subj E (k) lArr E (k minus 1) + ∆E y(k)

+ Dmin(k minus 1) minus ∆ P (k) ge 0 (24)

8 if E (k) gt E then

9

∆E x(k) lArr E (k) minus macrE E (k) lArr

macrE

10 else

11 ∆E x(k) lArr 012 end if

13 send (x(tk) ∆E x(k)) or discard14 end if

15 until termination

9

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 24: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2473

Figure 21 Energetics of the passive set-point modulation

D(k) =int tk+1tk

||x||2B and define modulated energy jump ∆P (k) as

∆P (k) = (12)||x(tk) minus y(k)||2K minus (12)||x(tminusk ) minus y(k minus 1)||2K

where || ⋆ ||K =radic

⋆T K⋆ and y(k) is the modulated version of y(k) via PSPM PSPM

modulates y(k) to y(k) in such a way that y(k) is as close to y(k) as possible (23)

while the energy jump ∆P (k) is limited by available energy in the system (24) Here

the available energy at time tk is the sum of E (kminus1) ∆E y(k) and Dmin(kminus1) where

E (k minus 1) is the energy left in the energy reservoir ∆E y(k) the shuffled energy from

peer PSPM and Dmin(k minus 1) a (substantial) portion of recycled damping dissipation

of B during [tkminus1 tk) Step 8-13 in Algorithm 1 define energy ceilingshuffling where

the energy reservoir E (k) is ceiled by E and the excessive energy ∆E x(k) is returned

to the peer PSPM or discarded if no peer exists See Fig 21 for energetics of PSPM

10

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 25: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2573

Using the modified control input (ie τ = minusBxminusK (xminusy(k))) for system (21) we

have the following inequality which will be used later in this paper forallT isin [tN tN +1)

991787 T

0

f T xdt = V (T )

minusV (0)

+N minus1sumk=0

D(k) minusN sumk=1

∆P (k) +

991787 T tN

||x||2B

ge V (T ) minus V (0) +N sumk=1

[Dmin(k minus 1) minus ∆P (k)]

= V (T ) minus V (0) + E (N ) minus E (0)

+N sumk=1

∆E x(k) minusN sumk=1

∆E y(k)

(25)

where V (T ) = (12)||x||2M (x) + (12)||x(T ) minus y(N )||2K and the last equality is due to

Dmin(k minus 1) minus ∆P (k) = E (k) minus E (k minus 1) + ∆E x(k) minus ∆E y(k) (26)

which can be obtained through step 7-12 in Algo1

11

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 26: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2673

Chapter 3

Wheeled mobile robot tele-driving

over the internet

31 Problem Setting of WMR Tele-Driving

We use a differential wheeled robot see Fig 31 as the slave wheeled mobile robot

(WMR) Denote its 3-DOF configuration by (xyφ) isin SE (2) with (x y) isin E (2)

being the WMR geometric center position and φ isin S the heading angle wrt the

global frame (OXY ) then the no-slip constraint can be written as

d

dt

x

y

φ

=

cos φ 0

sin φ 0

0 1

ν

ω

(31)

where ν isin real is the forward velocity of (x y) and ω = φ is the angular velocity When

the driven wheels accept torque as the input WMRrsquos dynamics can be written as Lee

(2007) m 0

0 I

ν

ω

+

0 minusmd φ

md φ 0

ν

ω

= u + δ (32)

12

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 27: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2773

Figure 31 Dynamic WMR tele-driving (q 1 ν )(q 2 φ) mode To achieve car driving metaphor we use q 1 to control WMRrsquos forward motion (ν ) and q 2 WMRrsquos turningmotion (φ or φ)

where I = I c + md2 with m I c gt 0 being the mass and moment of inertia wrt the

center of mass d is the distance between the geometric center and the mass center

and u = [uν uω]T δ = [δ ν δ ω]T are the control and the external forcetorque For

simplicity in this paper we assume d asymp 0 or we can cancel out the Coriolis terms in

(32) via a certain local control Then its reduced dynamics can be given by

m 0

0 I

ν

ω

= u + δ (33)

We call the above WMR described by (31) and (33) dynamic WMR On the other

hand many commercial WMRs (eg Pioneer DX-3 or e-puck) only accept ν ω as the

input signal that is its evolution is given by (31) and the following input equation

ν

ω

= u (34)

where u = [uν uω]T is the input signal If WMR can be written by (31) and (34)

we call it kinematic WMR

13

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 28: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2873

We want to achieve car-driving metaphor Diolaiti and Melchiorri (2002) Lee et al

(2002 2006b) masterrsquos one-DOF serves as the gas-pedal to command ν while the

other-DOF as the steering-wheel to command φ (or φ) For this we use the following

2-DOF linear master-joystick

h1q 1 = c1 + f 1 h2q 2 = c2 + f 2 (35)

where hi q i ci f i isin real are the mass configuration control and human force

In this paper we assume that suitable motionpower scaling factors as in Lee

and Li (2003 2005) are embedded in equations (31)-(35) to solve the mismatch in

the scale between the master device and WMR We want to coordinate q 1 with ν

and q 2 with φ (or φ) for both dynamic and kinematic WMR Then we can think

of the four modes of tele-driving 1) dynamic WMR (q 1 ν )(q 2 φ) tele-driving 2)

dynamic WMR (q 1 ν )(q 2 φ) tele-driving 3) kinematic WMR (q 1 ν )(q 2 φ) tele-

driving and 4) kinematic WMR (q 1 ν )(q 2 φ) tele-driving For each of them we

design PSPM-based tele-driving control laws Before doing that we briefly review

the PSPM framework

32 Dynamic WMR Tele-Driving Control

321 (q 1 ν )(q 2 φ) Tele-Driving

We first consider the (q 1 ν )(q 2 φ) tele-driving of the dynamic WMR For this we

extend the result of Lee (2008) by incorporating the scaling ρs of the master-slave

PSPM power shuffling This turns out to be crucial if the WMR has substantial

dissipation for which if not scaled up the virtually shuffled human power via PSPM

is simply all dissipated thus cannot drive the WMR Although our derivation here is

similar to that of Lee (2008) we include here since it will be used in the later sections

of this paper

14

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 29: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 2973

Following Lee (2008) we use the following tele-driving control

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus pν (k)) (36)

uν (t) =minus

bν (ν (t)minus

q 1(k)) (37)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (38)

uω(t) = minusbω φ(t) minus kω(φ(t) minus q 2(k)) (39)

where (36)-(37) are the tele-accelerating control while (38)-(39) tele-steering

control b⋆ k⋆ gt 0 are gains and ⋆(k) is the modulated version of ⋆(k) via the PSPM

We use a different definition for pν instead of pν = ν minus δ ν bν in Lee (2008) here

= ν + δ ν

We will explain this change after Th 31

Here note that we use PSPM for c1 c2 uw Consequently we will have two-port

passivity for (q 2 φ) mode On the other hand passivitystability combination will be

achieved for the (q 1 ν ) mode This is because the q 1 q 2 φ all are under the second-

order dynamics while ν under the first-order dynamics As we do not use PSPM for

uν the PSPM for c1 will discard excessive energy and not receive shuffled energy from

the WMR side Thus we will use the power-shuffling scaling ρs gt 0 only between c2

and uω that is instead of (24) we will have for c2

E 2(k) lArr E 2(k minus 1) + ∆E ω(k)ρs

+ D2min(k minus 1) minus ∆P 2(k) ge 0 (310)

while for uw

E ω(k) lArr E ω(k minus 1) + ρs∆E 2(k)

+ Dωmin(k minus 1) minus ∆P ω(k) ge 0 (311)

where ∆E ω(k)ρs and ρs∆E 2(k) represent the scaled power-shuffling from the WMR

and from the master respectively

15

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 30: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3073

Theorem 31 Consider the master (35) and the dynamic WMR (33) under the

control (36)-(39) Suppose there is no data duplication Lee and Huang ( 2009b)

Then the followings hold

1 The closed-loop q 1-dynamics is passive ie forallT ge 0 exist a bounded c isin real st

991787 T 0

f 1 q 1 ge minusc2 (312)

Also if the human operator is passive and the slave environmentrsquos instantaneous

power is bounded ie forallT ge 0 exist bounded constants α1 αv isin real st

991787 T

0

f 1 q 1 le

α21 δ v(T )ν (T )

leα2v (313)

ν -dynamics is stable in the sense of bounded ν (t) On the other hand the

closed-loop (q 2 φ)-system is two-port passive forallT ge 0 exist a bounded d isin real st

991787 T 0

(ρsf 2 q 2 + δ ωω)dt ge minusd2 (314)

2 Suppose that E 1(k) gt 0 forallk ge 1 (ie enough energy for c1 PSPM) Then i) if

(q 1 q 1 ν δ ν ) rarr 0 f 1 rarr k0ν or ii) if (q 1 q 1 ν ν ) rarr 0 f 1 rarr minus(k0 + 2k1)δ ν bν

3 Suppose that E 2(k) gt 0 and E ω(k) gt 0 forallk ge 1 (ie enough energy for c2 uw

PSPM) Then i) if (f 2 δ ω) = 0 φ rarr q 2 and ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarrminusk2kωδ ω

Proof We have the following closed-loop dynamics with the control (36)-(39)

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus pν (k)) = f 1 (315)

mν + bν (ν minus q 1(k)) = δ ν (316)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (317)

I φ + bω φ + kω(φ minus q 2(k)) = δ ω (318)

16

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 31: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3173

For the q 1-dynamics similar to (25) considering no energy shuffling for a single

PSPM we have forallT isin [tN tN +1) st

991787 T

0

f 1 q 1dt

geV 1(T )

minusV 1(0) + E

1(N )

minusE

1(0) +

N

sumi=1

∆E 1(i) (319)

where V 1(t) = 12

h1 q 21 + 12

k0q 21 + 12

k1(q 1 minus pν (k))2 This proves the passivity of the

q 1-dynamics with c2 = V 1(0) + E 1(0) The boundedness of q 1 q 1 and (q 1 minus pν ) can

also be shown from (319) with (313) Also from (316) we have

dκν

dt = minusbν ν 2 + bν q 1(k)ν + δ ν ν

where κν = mν 22 With the boundedness of q 1(k) (from (319) with |q 1(k)| le λ1)

and (313) we havedκν

dt le minusbν |ν |2 + bν λ1|ν | + α2

ν (320)

implying that |ν (t)| is ultimate bounded Khalil (1996) (ie |ν (t)| le max(|ν (0)| (bν λ1+radic b2ν λ

21 + 4bν α2

ν )(2bν ))

For the two-port passivity of (c2 uw) similar to (25) we can show that forallT ge 0

existN

1 N

2 st

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs

991787 T 0

δ ωωdt ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i)

17

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 32: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3273

where V 2(t) = 12

h2 q 22 + 12

k2(q 2 minus φ(k))2 and V ω(t) = 12

I φ2 + 12

kω(φ minus q 2(k))2 Also

with the no data duplication assumption

N 1

sumi=1

∆E 2(i)ge

N 2

sumi=1

∆E 2(i)N 2

sumi=1

∆E ω(i)ge

N 1

sumi=1

∆E ω(i) (321)

and combining these four inequalities we obtain

ρs

991787 T 0

f 2 q 2dt +

991787 T 0

δ ωωdt

ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves (314) with d2 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0) This proves the

two-port passivity with the scaling ρs gt 0 of the PSPMrsquos power-shuffling

For the second item when enough energy exists in the reservoir (ie pν = pν ) if

(q 1 q 1 ν δ ν ) rarr 0 considering pν = ν + δ ν bν rarr ν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus ν (k)) 0 rarr bν (ν minus q 1(k)) (322)

so we have f 1 rarr k0ν (the linear velocity perception) If (q 1 q 1 ν ν ) rarr 0 considering

pν = ν + δ ν bν rarr δ ν bν (315)-(316) reduce to

f 1 rarr k0q 1 + k1(q 1 minus δ ν (k)bν ) δ ν rarr minusbν q 1(k) (323)

so we have f 1 rarr minus(k0 + 2k1)δ ν bν (the force reflection)

The proof for the third item is exactly the same as that in Lee (2008) so omitted

here

Here a large ρs gt 0 would be desirable if the slave WMR is large or operating in

a highly dissipative environment This ρs may also be adapted on-line by monitoring

18

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 33: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3373

energy shuffling between the two systems although its detailed exposition we spare

for a future publication

Note that pν provides information for operators to perceive both the WMR velocity

ν and environment force δ ν The reason why we choose pν = ν + δ ν bν instead of

pν = ν minus δ ν bν Lee (2008) is that this pν can give an intuitive haptic perception

during the transient state when WMR is affected by the environment force δ ν For

instance if we keep q 1 gt 0 still (ie q 1 q 1) and WMR is running steadily (ν = 0)

toward the obstacle Suppose the environment force δ ν is rendered by some potential

field method and δ ν lt 0 decreases as WMR is getting close to the obstacle we have

velocity ν gt 0 decreasing when WMR enters the potential field and people can feel

this resistance (ie increasing f 1) via the decreasing pν = ν + δ ν If pν = ν minus

δ ν from

(316) we have pν = ν minus δ ν bν = q 1 minus mνbν As ν lt 0 decreases when WMR enters

the potential field and thus pν increases operator would have a confusing haptic

perception (f 1 gt 0 decreases when get close to an obstacle)

Note also that the signal pν for the c1rsquos PSPM is not purely a position signal but

rather a combination of force and velocity information This allows us to seamlessly

change haptic feedback mode between velocity feedback and force feedback as in item

2 of Th 31 which corresponds the following two scenarios cruising and hard-contact

322 (q 1 ν )(q 2 φ) Tele-Driving

Instead of q 2 rarr φ in Sec321 in this tele-driving control mode we want q 2 rarr φ

which is more similar to usual car driving Generally speaking q 2 rarr φ is suitable

when we hope to keep some direction wrt the global frame and q 2 rarr φ is more

suitable when we want to drive freely (eg turning around) Noting the analogy

between q 2 rarr φ problem and q 1 rarr ν problem we propose the following control

19

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 34: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3473

design instead of (38)-(39)

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus pω(k)) (324)

uω(t) =minus

bω(ω(t)minus

q 2(k)) (325)

where pω(k) is the modulated version of pω = ω minus δ ωbω via PSPM For tele-

accelerating control we still use (36)-(37) so the same result holds for the (q 1 ν )

tele-driving mode as in 321

Theorem 32 Consider the master device (35) and dynamic WMR (33) under the

tele-steering control (324)-(325)

1 The closed-loop q 2-dynamics is passive similar to (312) If the human operator

is passive and the slave environmentrsquos instantaneous power (ie δ ωω) is bounded

similar to (313) ω-dynamics is stable with bounded ω(t)

2 Suppose that E 2(k) gt 0 forallk ge 1 (ie enough energy for c2 PSPM) Then i) if

(q 2 q 2 φ δ ω) rarr 0 f 2 rarr kprime

0ω or ii) if (q 2 q 2 φ φ) rarr 0 f 2 rarr minus(k0prime + 2k2)δ ωbω

Proof We have the closed loop equations for the q 2-dynamics and ω-dynamics

h2q 2 + b2 q 2 + kprime

0q 2 + k2(q 2 minus pω(k)) = f 2 (326)

I φ + bω(ω minus q 2(k)) = δ ω (327)

Since (326)-(327) have the same form as (315)-(316) we can similarly prove the

passive q 2-dynamics and stable ω-dynamics as in Th 31

For the second item when enough energy exists in the reservoir (ie pω = pω) if

(q 2 q 2 φ δ ω) rarr 0 considering pω = ω + δ ωbω rarr ω (326)-(327) reduce to

f 2 rarr kprime

0q 2 + k2(q 2 minus ω(k)) 0 rarr bω(ω minus q 2(k)) (328)

20

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 35: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3573

so we have f 2 rarr kprime

0ω (the angular velocity perception) If (q 2 q 2 φ φ) rarr 0

considering pω = ω + δ ωbω rarr δ ωbω (326)-(327) reduce to

f 2 rarr

kprime

0q 2 + k2(q 2minus

δ ω(k)bω) δω

rarr minusbωq 2(k) (329)

so we have f 2 rarr minus(kprime

0 + 2k2)δ ωbω (the torque reflection)

Note that via pν and pω we can perceive the environment forcetorque explicitly

via δ ν δ ω being rendered by sensors or implicitly through ν ω changing in response

of the environment (eg stuck by an obstacle)

33 Kinematic WMR Tele-Driving Control

331 (q 1 ν )(q 2 φ) Tele-Driving

From (34) since we want q 1 rarr ν and q 2 rarr φ we can think of the control uν = q 1(k)

and use ν φ as the set-position signals for controlling q 1 and q 2 while modulating

these signals to guarantee passivity Based on this observation we define the control

law for (q 1 ν )(q 2 φ) tele-driving mode st

c1(t) = minusb1 q 1(t) minus k0q 1(t) minus k1(q 1(t) minus ν (k)) (330)

uν (t) = q 1(k) (331)

c2(t) = minusb2 q 2(t) minus k2(q 2(t) minus φ(k)) (332)

uω(t) = minuskω(φ(t) minus q 2(k)) (333)

where ⋆(k) is the modulated version of ⋆(k) via the PSPM

We extend the original PSPM approach (which only applies to second order

systems) for the first order system here Although we do not have energy definition

for kinematic systems we can build a storage function (similar to spring energy) for

the controller (333) as V ω(t) = 12

kω(φ(t) minus q 2(k))2 and define the energy jump at tk

21

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 36: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3673

by

∆P ω(k) = V ω(tk) minus V ω(tminusk ) (334)

Considering (333) and (34) we have forallt isin [tk tk+1)

dV ωdt

= kω(φ minus q 2(k)) φ = minusφ2 le 0

which shows that the storage is decreasing during each interval so we can express

the loss of storage Dω(k minus 1) during [tkminus1 tk) by

Dω(k minus 1) = V ω(tkminus1) minus V ω(tminusk ) =

991787 tktkminus1

φ2dt (335)

which is similar to damping dissipation Then we can apply PSPM approach to this

system with the energy jump (334) and the dissipation (335) just as we did for a

second order system with spring energy jump and damping dissipation

The following theorem summarize the main properties of this tele-driving control

law (330)-(333) In contrast to the conventional tele-operation system with the

WMR being first-order kinematic the closed-loop system is passive in the master

port while stable for the WMR port with passive human assumption

Theorem 33 Consider the master device (35) and the kinematic WMR (34)

under the tele-driving control (330)-(333) Suppose that there is no data duplication

Then the followings hold

1 The closed-loop q 1-dynamics and q 2-dynamics are passive ie forallT ge 0 existbounded d1 d2 isin real st

991787 T 0

f 1 q 1 ge minusd21

991787 T 0

f 2 q 2 ge minusd22

Also if the human operator is passive (313) ν ω-dynamics are stable with

bounded ν (t) and ω(t)

22

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 37: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3773

2 Suppose that E 1(k) gt 0 forallk ge 1 (enough energy in c1 PSPM) Then if (q 1 q 1) rarr0 f 1 = k0ν

3 Suppose E 2(k) gt 0 E ω(k) gt 0 forallk ge 1 (enough energy for c2 uω PSPM) Then

if (f 2 φ) rarr 0 q 2 rarr φ

Proof We have the closed-loop q 1 q 2-dynamics st

h1q 1 + b1 q 1 + k0q 1 + k1(q 1 minus ν (k)) = f 1 (336)

h2q 2 + b2 q 2 + k2(q 2 minus φ(k)) = f 2 (337)

with

ν = q 1(k) φ = minuskω(φ minus q 2(k)) (338)

from (331) (333) Due to the PSPMs installed both for c1 c2 we can get the

following inequalities forallT ge 0 existN N 1 st

991787 T 0

f 1 q 1dt ge V 1(T ) minus V 1(0) + E 1(N ) minus E 1(0) +N sumi=1

∆E 1(i) (339)

991787 T 0

f 2 q 2dt ge V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0)

+N 1sumi=1

∆E 2(i) minusN 1sumi=1

∆E ω(i)ρs (340)

where V 1(t) = 12h1 q 21 + 1

2k0q 21 + 12k1(q 1 minus ν (k))2 and V 2(t) = 1

2h2 q 22 + 12k2(q 2 minus φ(k))2

Then (339) suggests passive q 1-dynamics with d21 = V 1(0)+E 1(0) and also implies

bounded ν with (313) For the controller uω installed with PSPM considering (26)and the definitions for ∆P ω(k) (334) and Dω(k minus 1) (335) we have forallT ge 0 existN 2

23

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 38: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3873

st

V ω(T ) minus V ω(0)

= [V ω(T ) minusN 2sumi=1

∆P ω(i) minus V ω(0)] +

N 2sumi=1

∆P ω(i)

= [V ω(T ) minus V ω(tN 2) minusN 2minus1sumi=0

Dω(i)] +N 2sumi=1

∆P ω(i)

le minus991787 T tN 2

φ2dt minusN 2sumi=1

[Dωmin(i minus 1) minus ∆P ω(i)]

le minusN 2sumi=1

[E ω(i) minus E ω(i minus 1) + ∆E ω(i) minus ρs∆E 2(i)]

= minusE ω(N 2) + E ω(0) minusN 2sumi=1

∆E ω(i) + ρs

N 2sumi=1

∆E 2(i) (341)

which can be rearranged to

0 ge V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

+N 2sumi=1

∆E ω(i) minus ρs

N 2sumi=1

∆E 2(i) (342)

With the no data duplication assumption adding (340) and (342) we have

ρs

991787 T 0

f 2 q 2dt ge ρs(V 2(T ) minus V 2(0) + E 2(N 1) minus E 2(0))

+ V ω(T ) minus V ω(0) + E ω(N 2) minus E ω(0)

which proves the passive q 2-dynamics with d22 = ρsV 2(0) + ρsE 2(0) + V ω(0) + E ω(0)

and also proves bounded V ω(t) bounded φ

minusq 2(k) and bounded ω(t) with the passive

human assumption

For the second item if there is enough energy in the reservoir E 1 ν (k) = ν (k)

From (34) and (331) we have ν = uν = q 1(k) Then if (q 1 q 1) rarr 0 equation (336)

gives f 1 rarr k0ν For the third item when enough energy exists in the reservoirs

24

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 39: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 3973

φ(k) = φ(k) and q 2(k) = q 2(k) if φ rarr 0 we have φ(k + 1) rarr φ(k) Following the

result in (Lee and Huang 2009b Th 1) we have q 2 rarr φ

For this mode we can choose ρs to be small since one could assume that the

environment force is zero for a kinematic WMR and that there is no energy disspation

to the environment

332 (q 1 ν )(q 2 φ) Tele-Driving

The (q 2 φ) tele-driving mode is similar to (q 1 ν ) tele-driving mode in 331 so we can

use (330)-(331) for tele-accelerating control and the following control for the (q 2 φ)

tele-driving mode

c2(t) = minusb2 q 2(t) minus kprime

0q 2(t) minus k2(q 2(t) minus ω(k)) (343)

uω(t) = q 2(k) (344)

The following theorem can be proved similarly to Th 33 so we omit its proof

here

Theorem 34 Consider the master device (35) and the kinematic WMR (34)

under the tele-steering control (343)-(344)

1 The closed-loop q 2-dynamics is passive and the ω-dynamics is stable with

bounded ω(t) under the passive human assumption (313)

2 Suppose E 2(k) gt 0 forallk ge 1 (enough energy in c2 PSPM) if (q 2 q 2) rarr 0 we

have f 2 = kprime

25

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 40: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4073

Chapter 4

Quad-rotor type UAV

tele-operation over the Internet

41 Problem Setting of UAV teleoperation

We consider a quadrotor type UAV evolving on SE (3) with the following translational

dynamics and attitude kinematics Hua et al (2009)

mx = minusλRe3 + mge3 + δ (41)R = RS (ω) (42)

where m gt 0 is the mass x isin real3 is the Cartesian position wrt the NED (north-

east-down) inertial frame with e3 representing its down-direction λ isin real is the thrust

along the body-frame down direction δ isin real is the Cartesian disturbance R isin SO(3)

is the rotational matrix describing the body NED frame of UAV wrt the inertial

NED frame ω = [ω1 ω2 ω3]isin real

3 is the angular velocities of the body frame relative

to the inertial frame expressed in the body frame J isin real3times3 is the inertia matrix

wrt the body frame g is the gravitational constant and S ( ⋆) real3 rarr so(3) is the

skew-symmetric operator defined st for a b isin real3 S (a)b = a times b

26

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 41: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4173

The control input for this quadrotor-type UAV are λ isin real and the angular rate

ω isin real3 Since the available control for the UAV is only 4-dimensional while its

evolution is in 6-dimensional space (SE (3) = E 3 times SO(3)) this implies the UAV is

not fully actuated More specifically its Cartesian motion (41) can be affected by the

thrust λ although its direction is coupled to the attitude kinematics (42) Usually if

we control the UAV through a transmitter lots of training is needed before operator

can smoothly control the UAV and trying to maintain its attitude would distract the

operator from the main tasks like surveillance or transportation For this we design

a back-stepping trajectory tracking controller so that the operator only need to care

about the Cartesian position of the UAV while the UAV automatically control the

attitude and follow the desired Cartesian position

42 Backstepping trajectory tracking control of

UAVs

421 Controller Design

By inspecting the translational dynamics (41) we first define the following desired

control ν

ν = minusmxd + mge3 + be + ke (43)

where xd(t) isin real3 is the desired trajectory e(t) = x(t) minus xd(t) is the tracking error

and b k gt 0 are dampingspring gains Our wish is to let the term λRe3 in (41)

to be the same as the virtual control ν yet we can not directly control the rotation

matrix R that is there would be some error betweeen them and define the control

generation error ν e st

ν e = λRe3 minus ν (44)

Here to derive our ldquonominalrdquo control we assume no Cartesian disturbance with δ = 0

in (41) we will analyze its effect in Sec 422

27

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 42: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4273

By (41) (43) and (44) we can write the closed-loop dynamics

me + be + ke = ν e (45)

for which we define the first Lyapunov function

V 1 = 1

2meT e + mϵeT e +

1

2(k + ϵb)eT e (46)

where ϵ gt 0 is a constant to be chosen below The term with ϵ is called cross-coupling

term to achieve exponential convergence for the above linear second-order dynamics

with ν e = 0 () Differentiate V 1 with (45) we then have

V 1 = minus(b minus mϵ)eT e minus ϵkeT e minus (e + ϵe)T ν 3 (47)

Here we define the following two matrix P Q and vector ζ st

P =

m ϵm

ϵm k + ϵb

otimes I 3 Q =

b minus ϵm 0

0 ϵk

otimes I 3 ζ =

e

e

(48)

with P Q ≻ 0 if 0 lt ϵ lt bm then we have

V 1 = ζ T P ζ V 1 = minusζ T Qζ minus (e + ϵe)T ν e (49)

From (49) if ν e = 0 we would have (e e) rarr 0 exponentially Aiming to bound

ν e about the origin we augment V 1 st

V 2 = V 1 + 1

2γ ν T e ν e (410)

28

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 43: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4373

where γ gt 0 is a constant which would be helpful for the robustness analysis in Sec

422 Differentiating V 2 we have

V 2 =

minusb

2

eT e

minusϵkeT e +

1

λ

ν T e (ν eminus

γ (e + ϵe)) (411)

If we use the following update law for ν e st

ν e = γ (e + ϵe) minus αν e (412)

we can have

V 2 = minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e (413)

implying exponential convergence of e e and ν e

The control design for (λ ω) is in fact From (42) and (412) we have

ν e + αν e = Rλe3 + Rλe3 minus ν + α(λRe3 minus ν ) = R

λω2

minusλω1

λ + αλ

minus ν minus αν (414)

from which and (412) we have the following control law in the following form

R

λω2

minusλω1

λ + αλ

= ν + αν + γ (e + ϵe) = ν or

λω2

minusλω1

λ + αλ

= RT ν (415)

where ν in ν can be computed by

ν = minusmx d minus bxd + ke +

b

m(minusλRe3 + mge3) (416)

From equation (415) we can get the control law for (λ ω1 ω2) as long as λ = 0 The

singularity at λ = 0 seems not likely happen in practice unless operator command

the UAV to ldquofree-fallrdquo (Frazzoli et al (2000))

29

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 44: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4473

The relation(415) also shows that any (smooth) desired control ν can be

incorporated into our backstepping control design as long as it produce a relation

similar to (415) and its computation is implementable similar to ν in (416) hear

Also note that the control parameters b k α have clear physical meanings thereby

making the tuning very intuitive These manifest the flexibility and transparency of

our backstepping control law Note also from (415) that in order to produce the

desired control ν we only need λ ω1 ω2 not ω3 In other words this ω3 is redundant

and we may simply set ω3 = 0 or use it for other purpose (eg to head to a certain

direction while flying) This also reaffirms the well-known fact (eg Aguiar and

Hespanha (2007)) that for a thrust-propelled rigid body on SE (3) we only need

one thrust force and two angular rates along the other two (body) axes to control

its Cartesian motion in E 3 The following Th 41 summarizes our backstepping

trajectory tracking control design and its key properties

Theorem 41 Consider the UAV (41)-(42) under the backstepping control (415)

with ω3 and xd xdx d being all bounded Suppose that existϵλ gt 0 stλ(t) ge ϵλforallt ge 0

Then (e eν e) rarr 0 exponentially and (x xλω) are bounded

Proof Exponential convergence of (e eν e) and their boundedness have already been

shown above λ is bounded from (44) with bounded ν e and ν ν is bounded from

(416) which implies ν is also bounded with its definition in (415) Thus from (415)

with the assumption that λ(t) ge ϵλ gt 0 ω1 ω2 are bounded Also the boundedness

of ω3 and x can be easily seen from ω3 = 0 and (45)

422 Robustness Analysis

In this section we will analyze the robustness of the backstepping control in the

presence of unknownbounded disturbance δ and inaccurate estimate of m (identified

wrt a certain unit of λ)

30

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 45: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4573

Here the desired control ν is given by

ν = minusmxd + mge3 + be + ke (417)

where m = m + m gt 0 is the estimate of the UAV mass m and instead of (45) the

closed-loop dynamics is given by

me + be + ke = minusν e minus d (418)

where d = minusmxd + mge3 minus δ is the combined (bounded) effect of the uncertain m

and the disturbance δ Using the same V 1 as in (49) we have

V 1 = minus b

2 eT e minus ϵkeT e minus (e + ϵe)T (ν e + d) (419)

and using the same V 2 as in (410) we have

V 2 = minus b

2 eT e minus ϵkeT e +

1

γ ν T e (ν e minus γ (e + ϵe)) minus (e + ϵe)T d (420)

where the last term cannot distabilize the system since d is bounded

The above inequality suggests that we can use the same update law (412) for

ν e This (412) may not appear to require the correct estimate of m Yet if we

inspect its decoding (415) and (416) which gives the actual control action (λ ω)

we will find that this update law (412) will require us to have a information of x

which is estimated by x = 1m

(minusλRe3 + mge3) Here note that even with uncertain

m ν e = λRe3 minus ν in (44) is still certain since λRe3 and ν are known (although not

necessarily accurate) If we use the possible incorrect estimate m for x which then

comes into ν (416) and ν (415) we end up with

ν e = γ (e + ϵe) minus αν e + emminus1bλRe3 minus b

mδ (421)

31

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 46: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4673

where emminus1 = (1m minus 1m) = m(mm) is due to the uncertainty in m

Inserting this (imperfect) update law into the above V 2 and using the definition

of ν e (44) and ν (417) we have

V 2 = minus b2

eT e minus ϵkeT e minus αγ

ν T e ν e + bemminus1

γ ν T e (ν e + ν ) minus b

γmν T e δ minus (e + ϵe)T d

= minus b

2 eT e minus ϵkeT e minus α

γ ν T e ν e minus (minusb2emminus1

γ )ν T e e minus (minusbkemminus1

γ )ν T e e

minus bemminus1

γ ν T e (mxd minus mge3) minus b

γmν T e δ minus (e + ϵe)T d

= minusζ T Qprime

ζ minus bemminus1

γ ν T e d minus (e + ϵe)T d

(422)

where ζ = [e e ν e] d =minus

mxd + mge3 + δ

memminus

1

and Qprime

is defined as

Qprime

=

b2

0 minus b2emminus1

0 ϵk minus bkemminus1

minus b2emminus1

2γ minus bkemminus1

αminusbemminus1

γ

Then following Horn and Johnson (2005) we can set Qprime ≻ 0 by choosing α γb k

st

α ge bemminus1 + b

2

e

2

mminus1

(2ϵb + k)4ϵγ (423)

This implies even with the uncertain m and unknownbounded δ the system will

still be stable with (e eν e) being ultimately bounded Khalil (1996)

Note that given the estimation error emminus1 and b k this condition (423) is always

grantable if we choose α and γ large enough Large α γ are possible as they only

affect the system behavior in software On the other hand the ultimate bound of

(e eν e) is likely shrinking with large b k Yet b k are not freely tunable because

large b k may cause the system to reach the limitation of hardware (ie actuator

saturation) It is also worthwhile to mention that in many cases it is not so difficult

to estimate m fairly precisely (eg less than 5 error for our experiment in Chapter

32

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 47: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4773

Figure 41 UAV teleoperation The velocity of the virtual point ˙ p is controlled by

the configuration of the hapitc device q and affected by the repulsion partϕT opartp

from thevirtual obstacle The UAV follows the virtual point through a backstepping controller

5) Note also that we can also try to cancel out the disturbance δ by putting an

estimate δ in ν (417) to enhance the control performance

43 Application to UAV teleoperation over the

Internet

431 Virtual Point and Virtual Environment

In this section we show how the backstepping control can be used for the UAV haptic

teleoperation over th Internet We consider a virtual point (VP) evolving according

to

˙ p = ηq (k) minus part ϕT 0

partp (424)

where p isin real3 is the VPrsquos position q (k) isin real3 is the master position received via

the Internet at time tk η gt 0 is to match different scales between q (t) and ˙ p and

ϕo(

|| p

minus po

||) is the obstacle avoidance potential producing repulsive force when

|| p minus po|| becomes small

The control ηq (k) enables the user to tele-control the VPrsquos velocity ˙ p by the

master devicersquos position q (t) thereby to circumvent the problem of master-slave

kinematic dissimilarity (ie stationary master with bounded workspace mobile VP

33

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 48: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4873

with unbounded workspace Lee et al (2006a)) The kinematic VP is also chosen

here in contrast to the second-orderdynamic VPs () since it greatly simplifies the

stability and collision avoidance analysis as shown below

Since we also need to compute ˙ p uml p p for UAV trajectory tracking control and

time derivative for p(k) is not well-defined we remedy this by changing (424) to the

following

˙ p = ηq (t) minus part ϕT o

partp (425)

where q (t) is defined st

umlq (t) + bprime

˙q + kprime

q (t) = a2q (k) (426)

ie second-order critically damped filter with a gt 0 bprime

= 2a kprime

= a2 Here note

that due to the second-order nature umlq (t) ˙q (t) q (t) are all bounded as long as q (k) is

bounded regardless of discontinuity of q (k) If we set a to be large enough we can

practically ensure ||q (k) minus q (k)|| to be small enough Since the VP (425) dynamics

is simulated in software we may indeed choose this a to be large enough if that is

desired Now we can compute uml p p as follows

uml p = minusH ϕo( p) ˙ p + η ˙q (t) (427)

p = minusdH ϕo( p)

dt ˙ p minus H ϕo uml p + η umlq (t) (428)

= minusdH ϕo( p)

dt ˙ p + H ϕo( p)[H ϕo( p) ˙ p minus η ˙q (t)] minus η[b

prime

˙q (t) + kprime

(q (t) minus q (k))]

where H ϕo( p) = [ part 2ϕo

partpipartpj] isin real3 is the Hessian of ϕo

Let us define a Lyapunov function to analyze the stability of the pminusdynamics and

q minusdynamics st

V p = ϕo + 1

2|| ˙q ||2 + ϵ ˙q T q +

1

2(k

prime

+ ϵbprime

)||q ||2 = ϕo + [ ˙q q ]T Q[ ˙q q ] (429)

34

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 49: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 4973

where ϵ gt 0 is choose by (431) and Q is defined as

Q =

12

12

ϵ

12 ϵ 1

2(kprime

+ ϵbprime

)

From (425) and (426) we have

dV pdt

= minusζ T Qζ minus uT ζ le minusσ[ Q] middot ||ζ ||2 + ||u| | middot | |ζ || (430)

where ζ = [partϕT o partp q ˙q ] and u = k

prime

q (k) middot [0ϵ 1] otimes I 3 and σ[ Q] is the minimum

singular value of Q with

Q =

1

minusη2

0

minusη2 ϵk

prime

0

0 0 bprime minus ϵ

Here we want Q Q ≻ 0 thus we can choose ϵ as follows

ϵ2 minus bprime

ϵ minus kprime

lt 0 ϵ lt b ϵkprime

gt η24

which can be simplified as

η2(4kprime) lt ϵ lt b prime (431)

The inequality (430) also implies that similar to the case of ultimate boundedness

if ||ζ || ge ||u||σ[Q] we will have V p le 0

Based on this observation we have the following Prop (1) for which we assume

that ϕo is constructed st 1) there exists a large enough M gt 0 st V (t) le M

implies no collision with the obstacle and 2) ||partϕopartp|| is bounded if ϕo(|| p minus po||) is

bounded

Proposition 1 Suppose q (k) is bounded forallk ge 0 Suppose further that if ϕo(|| p minus po||) ge M

||partϕo

partp || ge k

prime

q max

radic 1 + ϵ2

σ[ Q] (432)

35

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 50: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5073

Then ϕo le M forallt ge 0 and q ˙q umlq are bounded Suppose further that H ϕo dH ϕodt

are

bounded if ϕo le M Then ˙ p uml p p are all bounded

Proof The inequality implies that if ||ζ || ge ||u||σ[Q]

V p le 0 With the above condition

(432) if ϕo ge M we have

||ζ | |ge | |partϕo

partp || ge ||u||

σ[ Q] (433)

which further implies that

ϕo le V p(t) le M

and that partϕopartp

H ϕo dH ϕodt

are bounded The boundedness of q ˙q umlq is a direct

consequence of the mass-spring-damper type dynamics (426) and the boundedness

of ˙ p uml p

p can be deduced from equation (425) (427) and (428)

Now given that ˙ p uml p p are all well-defined the backstepping control can the

robustly enable the UAV to track the trajectory of the VP Here we assume the

obstacle is designed st it rapidly increases as the VP approaches the obstacle to

prevent collision while gradually converge to zero as || p minus po|| rarr 0 so that the effect

of the obstacle can smoothly emerges when they gets close to the VP

One possible potential function if we define ||x|| = || p minus po|| could be

ϕo(||x||) =

k( ||x||2minusd2

||x||2minusmicro2)2 micro lt ||x|| le d

0 ||x|| gt d

withpartϕo

partp = 4k(d2 minus micro2)

||x||2 minus d2

(||x||2 minus micro2)3( p minus po)T

where d gt micro gt 0 Other form of potential field (eg Ji and Egerstedt (2005)) is also

possible

36

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 51: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5173

432 Control Design for Haptic Device

For the haptic device control design we use passive set-position modulation

framework again to address the kinematicdynamic discrepancy between the master

device (fully-actuated Lagrangian dynamics with bounded workspace) and the VPs

(kinematic system with unbounded workspace) and to guarantee passivitystability

over the Internet with useful haptic feedback

We consider 3-DOF haptic device with the following dynamics

M (q )q + C (q q )q = τ + f (434)

where q isin real

3 is the configuration M (q ) isin real

3times3 is the positive-definitesymmetric

inertia matrix C (q q ) isin real3times3 is the Coriolis matrix and τ f isin real3 are the control

and human forces respectively For the control of the haptic device we first design

a feedback signal y(t) isin real3 received on the master side st

y(t) = 1

η(x minus part φT

o

partp ) (435)

where x is the UAVrsquos velocity in (41) and

minuspartφT opartp

is the virtual environment force

This feedback allows operator to perceive the state of the real UAV and to sense the

presence of the obstacle on VP

This y(t) is then sent back to the master side over the Internet Let y(k) denote

the signal received on the master side then similar to the control design in Chapter

3 we can design the control τ st for t isin [tk tk+1)

τ (t) = minusB q minus K 1q minus K (q minus y(k)) (436)

37

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 52: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5273

where B K 1 K ≻ 0 are diagonal gain matrices and y(k) is the PSPM modulated

version of y(k) st

minyk ||

y(k)

minusy(k)

|| (437)

subjE (k) = E (k minus 1) + Dmin(k minus 1) minus ∆ P (k) ge 0 (438)

Note in this case only one PSPM is used on the master side the excessive energy in

the PSPM would be discarded Now we can summarize the result in this chapter

Theorem 42 Consider the master device (434) with control (436) Then the

follows hold

1 The closed-loop master system is passive existc1 isin real stint T

0 f T q ge minusc21 forallT ge 0

Moreover if the human user is also passive (ie existc2 isin real stint T 0

f T qdt lec22 forallT ge 0) the closed-loop VPs teleoperation system is stable with q q q minus y(k)

and p being all bounded

2 Suppose q q rarr 0 E (k) ge 0forallk ge 0 and x rarr ˙ p Then a) if minus partϕT 0partp

= 0 (eg no

obstacles) f (t) = K 1η

x (ie velocity perception) or b) if x = 0 (eg stopped by

obstacles) f (t) rarr K 1+2K

η

partϕT o

partp (ie environment force perception)

Proof For the first item following the same procedure as in Lee and Huang (2009b)

we can show that forallT ge 0

991787 T 0

f T xdt ge V m(T ) minus V m(0) + E (N T ) minus E (0) (439)

where V m(t) = ||q ||2M 2+ ||q (tk)minusy(k)||2K 2+ ||q ||2K 1 and N T is the k-index happening

just before T From (439) we can show the master passivity with c2

1 = V m(0)+ E (0)and bounded q q q minus y(k) and p with the passive human assumption

38

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 53: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5373

For the second item if q q rarr 0 and E (k) ge 0forallk ge 0 (ie y(k) = y(k)) the

closed-loop master dynamic becomes

K 1q + K (q

minusy(k))

rarrf (t) (440)

with q (t) rarr q and minus partϕT 0partp

= 0 we have ˙ p rarr ηq rarr ηq (k) rarr ηq Thus if x rarr ˙ p we

have f (t) = K 1η

x If x = 0 thus p rarr 0 from the assumption and from (425) we have

0 = q minus part ϕT 0

partp y = minus1

η

partϕT 0

partp

then with (440) we have f (t) rarr K 1+2K η

partϕT opartp

39

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 54: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5473

Chapter 5

Experiment

51 WMR haptic tele-driving Experiment

We use a Phantom Desktop as the master device and a differential wheeled mobile

robot as the slave WMR see Fig 31 we use three ultrasonic rangefinders fixed at

the front of the WMR to render external repulsion (eg δ ν or δ ω) from obstacles

This virtual forcetorque will exert on the robot and also be perceived by human

(through pν = ν + δ ν bν or pω = ω + δ ωbω) The local servo-rates for the

haptic device and WMR are 1ms and 2ms respectively They are connected overWLAN (wireless local area network) with data buffering to set the communication

delay Two magnitudes of round-trip delay are considered randomly ranging from

025sec to 035sec (0125sim0175sec forth plus 0125sim0175sec back) corresponding to

intercontinental tele-operation over Internet Elhajj et al (2001)) and another ranging

from 1sec to 2sec (05sim1sec forth plus 05sim1sec back) The packet-loss is around 90

and packet-to-packet separation time is 15sim300ms with an average of about 50ms

We use linearangular motion scaling η1η2 and energy scaling ρ to scale the hapticdevice side to match the mechanical scale of the WMR Energy shuffling scaling ρs

is also used to enlargeshrink energy shuffled between haptic device and WMR if

PSPM is used on both sides

40

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 55: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5573

minus1

0

1Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

0 2 4 6 8 10minus10

0

10Angle Velocity Haptic Feedback

f 2

[ N ]

time [sec]

Figure 51 Unstable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

minus05

0

05Linear Velocity Coordination

q 1

η 1 ν

[ m s ]

q 1η1ν

minus10

0

10Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ

[ r a d ]

q 2η2

θ

0 1 2 3 4 5 6minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

time [sec]

Figure 52 Unstable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

without using PSPM

Experiments are shown for tele-driving with 025sim035sec randomly varying delay

in dynamic WMR (q 1 ν )(q 2 φ) mode and kinematic WMR (q 1 ν )(q 2 φ) mode

without using PSPM as shown in Fig 51 Fig 52 We pull the haptic device with a

moderate force at 25sec 2sec and release it then the systems become unstable Then

we make the same experiments when PSPM is utilized As shown in Fig 53 Fig

54 after the disturbance given from the master side the systems are stabilized and

(q 1 ν ) (q 2 φ φ) are coordinated Note the received signals are modified by PSPM to

avoid the accumulation of energy jump whenever the energy in the reservoir depletes

Also by inspecting the changing of energy reservoirs in Fig 54 we can see the (q 1 ν )

control is more likely to be destabilized than (q 2 φ) control with certain controller

parameters as in our case Also we can see from Fig 52 and Fig 54 that q 2 does

not return to zero This is due to the friction for the q 2-DOF on the master side

Now we test the dynamic WMR (q 1 ν )(q 2 φ) tele-driving The WMR travels

around two obstacles in an rsquo8rsquo figure shape see fig 55 with randomly varying

delay 025sim035sec and 1sim2sec (Fig 56 and 57 respectively) The WMR first

travels forward to the top of the rsquo8rsquo figure and travels backward to the starting

point As predicted in Th 31 1) the tele-operation is stable even with round-trip

41

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 56: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5673

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10

Linear Velocity Haptic Feedback

1

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus5

0

5Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 2 4 6 80

05

1

E 2

E 2

time [sec]

Figure 53 Stable case dynamicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

minus1

0

1Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus10

0

10Linear Velocity Haptic Feedback

1

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

005

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 2 4 6 80

05

1

E ω

E ω

time [sec]

Figure 54 Stable case kinematicWMR (q 1 ν )(q 2 φ) tele-driving with025sim035sec randomly varying delay

1sim2sec randomly varying delay 2) linear velocity (ν ) follows after the haptic device

configuration q 1 3) people can perceive the linear velocity ν via the local spring k0

and 4) (q 2 φ) coordination is achieved after operator releasing the device(eg after

50sec in Fig 57) Note that the tracking error in (q 1 ν ) coordination (eg 5-15sec

in Fig 56) is due to the friction pointing backward and that the communication

delay causes bumps for f 1 (eg around 4sec in Fig 56) and for f 2 (around 48sec in

Fig 57)

In Fig 58 the operator commands the WMR to move toward the wall with a

moderate constant q 1 The WMR measures distance from the wall using sonar sensors

and stops in front of the wall (around 11sec) then after the operator releasing the

device the WMR is bounced back (after 16sec) The operator can feel the approach

42

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 57: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5773

Figure 55 Travel around two obstacles in an rsquo8rsquo figure shape

of obstacles via f 1 which jumps (around 13sec) while q 1 being kept still Besides the

third subplot shows the energy changing in the energy reservoir E 1 Note that some

energy in the local spring flows to the reservoir after haptic device being released

(15-20sec)

For dynamic WMR (q 1 ν )(q 2 φ) tele-driving the WMR shows a similar per-

formance (see Fig 59 Fig 510) except that here q 2 and φ are coordinated

Also the operator can perceive the angular velocity via the local spring Note

when operator tele-drives WMR in (q 1 ν )(q 2 φ) mode operator tries to keep WMR

running forward while tele-driving WMR in (q 1 ν )(q 2 φ) mode if the heading angle

is large command WMR to turn around and run backward would become more

convenient (eg Fig 56)

In another experiment shown in Fig 511 we try to drive the WMR along the wall

toward a corner with moderate q 1 and q 2 due to the virtual forcetorque produced

from the wall the WMR flees away from the corner (around 9-14sec) even with q 2

being kept still and the operator can feel the increase and decrease in the resistant

forcetorque (around 12-17sec) Also the last two subplots show the energy changing

in reservoir E 1 and E 2

43

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 58: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5873

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 400

05

1

E ω

E ω

time [sec]

Figure 56 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delayAverage packet-to- packet interval2513ms for the haptic device and

2963ms for the WMR

minus05

0

05

inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 57 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2sec randomly varying delay Averagepacket-to- packet interval 5077ms forthe haptic device and 9246ms for the

WMR

Shown in Fig 512 and Fig 513 are the experimental results for the kinematic

WMR (q 1 ν )(q 2 φ) tele-driving As predicted in Th 33 1) the system shows a

stable behavior 2) the operator have velocity perception via the local spring k0 and

3) the coordination of (q 2 φ) is achieved after the operator releasing the device (eg

after 43sec in Fig 512) Note the haptic feedback f 2 caused by the tracking error

(eg 15-20sec in Fig 513) serves as a helpful indicator of the (q 2 φ) coordinating

process Note that there are lots of oscillation of q 1 or q 2 in Fig 515 This is due

to the operatorrsquos overcompensation when trying to maintain the WMR on the right

routine

44

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 59: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 5973

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1ν

minus2

02

Linear Velocity Haptic Feedback

f 1

[ N ]

0 5 10 15 20 250

05

1

Energy Changing in the Reservoir E1

E 1

E 1

time [sec]

Figure 58 Dynamic WMR (q 1 ν )(q 2 φ) hard contact

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 500

05

1

E 2

E 2

time [sec]

Figure 59 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with025-035sec randomly varying delay

Average packet-to- packet interval2549ms for the haptic device and3032ms for the WMR

minus05

0

05 Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 510 Dynamic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Average

packet-to- packet interval 4931ms forthe haptic device and 8678ms for theWMR

45

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 60: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6073

minus05

0

05Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus1

0

1Angle Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus05

0

05Angle Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 5 10 15 20 250

05

1

E 2

E 2

time [sec]

Figure 511 Flee Away from the Corner for dynamic WMR (q 1 ν ) = (q 2 φ)running into a corner (around 9 sec) q2 is kept but the WMR steers clear of theobstacle (during 9-14 sec)

46

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 61: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6173

minus05

0

05inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus2

0

2Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus1

0

1Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 500

05

1

E ω

E ω

time [sec]

Figure 512 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2472msfor the haptic device and 2895ms for

the WMR

minus02

0

02inear elocity oordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus5

0

5Linear Velocity Haptic Feedback

f 1

[ N ]

minus5

0

5Heading Angle Coordination

q 2

η 2

θ [ r a d ]

q 2η2

θ

minus2

0

2Heading Angle Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0

05

1

E 2

E 2

0 10 20 30 40 50 600

05

1

E ω

E ω

time [sec]

Figure 513 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5044ms forthe haptic device and 8643ms for the

WMR

In the last two Fig 514 and Fig 515 we use the control law in Th 34 to tele-

drive the kinematic WMR both velocity (ν φ) and haptic feedback (f 1 f 2) follow

tightly after (q 1 q 2) And also when the delay is high people spent more time to

finish the task

52 UAV haptic tele-operation

For UAV haptic teleoperation we also use Phantom hapitc device (from Sensable

update rate 1kHz as the master device and a Hummingbird quadrotor (from

Ascending Technologies) as the slave robot A regular PC (from Dell) with 186GHz

47

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 62: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6273

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 400

05

1

E 2

E 2

time [sec]

Figure 514 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 025-035sec randomly varying delay Aver-age packet-to- packet interval 2360msfor the haptic device and 1939ms forthe WMR

minus02

0

02Linear Velocity Coordination

q 1

η 1

ν [ m s ]

q 1η1

ν

minus2

0

2Linear Velocity Haptic Feedback

f 1

[ N ]

minus05

0

05Angular Velocity Coordination

q 2

η 2

˙ θ [ r a d s ]

q 2η2

θ

minus2

0

2Angular Velocity Haptic Feedback

f 2

[ N ]

0

05

1

Energy Changing in the Reservoirs

E 1

E 1

0 10 20 30 40 50 600

05

1

E 2

E 2

time [sec]

Figure 515 Kinematic WMR(q 1 ν )(q 2 φ) tele-driving with 1-2secrandomly varying delay Averagepacket-to- packet interval 5114ms forthe haptic device and 1838ms for theWMR

CPU and 2G memory is used as the server of the system On the PC we run thebackstepping controller (update rate 50Hz) and simulate the virtual environment

(update rate 100Hz) The PC and UAV are connected through a Bluetooth

communication and PC sends command at the rate of 50Hz the same as the

backstepping controller update rate A Vicon Vision Tracking System (from Vicon)

is running at 100Hz (limited by the computation capability of the computer) with a

latency of 200ms (also limited by the PC) to capture the motion of the UAV From

our control frame work the backstepping controller and the virtual environment isrunning on the slave side Here for experimental purpose they are both running on

the local PC yet we use some data buffers to created delays as if the backstepping

controller and virtual environment is on the remote side

48

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 63: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6373

Figure 516 Context diagram for the UAV haptic teleoperation program

To implement the above controller I wrote the multithread C++ program on

Windows The program runs the trajectory tracking controller simulates the virtual

environment rendering haptic force and regulates the data flow between PC haptic

device vision tracking system and UAV This is illustrated in the following context

diagram Fig 516

First we test our backstepping controller and try to let the UAV travel

autonomously along a predefined ldquo8rdquo figure shape The UAV starts from hovering

state and tries to follow the trajectory after 0sec As shown in the Fig 517 the

trajectory tracking is achieved with a low tracking error

||x

minusxd

||

Then we teleoperate the UAV fly towards a virtual obstacle placed at x = [0 15 0]

which is marked by an actual box The virtual point is prohibited from entering the

virtual obstacle which in turn prohibits the UAV from running into the actual box

As we can see from Fig 518 the UAV follows the desired trajectory well at the

beginning and operator can perceive the velocity of the UAV (from 0-7sec) As the

UAV approaches the obstacle at around 7sec the force feedback along x2 direction

suddenly increases which give the operator clear perception of the emergence of

environment obstacle After 12sec when the UAV flies out of the range of virtual

force we can see the human operator is able to perceive the velocity of the UAV

again

49

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 64: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6473

minus1minus05

005

1

minus1

0

1

minus2

minus15

minus1

minus05

0

x2 [m]

Trajectory Tracking

x1 [m]

x 3

[ m ]

Desired trajectory

UAV trajectory

0 5 10 15 200

05

1

15Tracking Error

t [sec]

x

minus

x d

[ m ]

t=9

t=3

t=2

Figure 517 UAV trajectory tracking along a predefined ldquo8rdquo figure shape Thetrajectory in this case is xd = [05 cos(02πt) 05sin(04πt) 12 minus 05cos(02πt)

minus1 minus05 0 05 1 15minus2

0

2

x1 [m]

x2 [m]

VP and UAV Trajectory

x 3

[ m

]

VP trajectory

UAV trajectory

0

02

04Tracking Error

t [sec]

x

minus

x d

[ m ]

0 5 10 15 20minus15

minus1

minus05

0

05

1

15

˙ x 2

[ m s ]

t [sec]

Haptic Perception of Obstacle and Velocity

0 5 10 15 20

minus4

minus2

0

2

4

f [ N ]

x2

f

t=0 t=7

t=16

Figure 518 UAV haptic teleoperation over the Internet with randomly varyingdelay 025sec minus 035sec

50

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 65: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6573

Chapter 6

Conclusions

In the thesis we study the controller design for the wheeled mobile robot(WMR)and unmanned aerial vehicle(UAV) haptic teleoperation over the Internet Both

dynamickinematic WMR are considered and a kind of thrust propelled UAV is

considered We extend the recently proposed passive-set-position-modulation to

settle the problem of instability induced by the varying-delay and packet-loss in the

communication For UAV teleoperation we also derive a backstepping trajectory

tracking control with robustness analysis Experiments for WMRs and UAV haptic

teleoperation over the Internet are shown to prove the efficacy of the controlframework This thesis would serve as a good guide for controller design for the

mobile robot haptic teleoperation

51

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 66: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6673

Bibliography

52

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 67: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6773

Bibliography

Aguiar A and Hespanha J (2007) Trajectory-tracking and path-following

of underactuated autonomous vehicles with parametric modeling uncertainty

Automatic Control IEEE Transactions on 52(8)1362ndash1379 6 30

Anderson R and Spong M (1989a) Bilateral control of teleoperators with time

delay Automatic Control IEEE Transactions on 34(5)494ndash501 2

Anderson R and Spong M (1989b) Bilateral control of teleoperators with time

delay IEEE Transactions on Automatic Control 34(5)494ndash501 5

Baiden G and Bissiri Y (2007) High bandwidth optical networking for underwater

untethered telerobotic operation In OCEANS 2007 pages 1ndash9 1

Berestesky P Chopra N and Spong M (2004a) Discrete time passivity in bilateral

teleoperation over the internet In Robotics and Automation 2004 Proceedings

ICRArsquo04 2004 IEEE International Conference on volume 5 pages 4557ndash4564

IEEE 2

Berestesky P Chopra N and Spong M (2004b) Discrete time passivity in bilateral

teleoperation over the internet In Proc IEEE Int Conf Robot Autom pages 4557

ndash 4564 5

Blthoff D L A F P R H I S H H (2010) Haptic Teleoperation of Multiple

Unmanned Aerial Vehicles over the Internet In Robotics and Automation (ICRA)

2011 IEEE International Conference on page To appear 3

53

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 68: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6873

Bouabdallah S and Siegwart R (2005) Backstepping and sliding-mode techniques

applied to an indoor micro quadrotor In Robotics and Automation 2005 ICRA

2005 Proceedings of the 2005 IEEE International Conference on pages 2247ndash2252

IEEE 6

Castillo P Dzul A and Lozano R (2004) Real-time stabilization and tracking of

a four-rotor mini rotorcraft Control Systems Technology IEEE Transactions on

12(4)510ndash516 6

Cho S K Jin H Z Lee J M and Yao B (2010) Teleoperation of a mobile

robot using a force-reflection joystick with sensing mechanism of rotating magnetic

field Mechatronics IEEEASME Transactions on 15(1)17 ndash26 1

Chopra N Spong M Hirche S and Buss M (2003) Bilateral teleoperation over

the internet the time varying delay problem In American Control Conference

2003 Proceedings of the 2003 volume 1 pages 155 ndash 160 2

Diolaiti N and Melchiorri C (2002) Teleoperation of a mobile robot through haptic

feedback IEEE HAVE pages 67ndash72 3 4 14

Elhajj I Xi N Fung W Liu Y Li W Kaga T and Fukuda T (2001)Haptic information in internet-based teleoperation IEEEASME Transactions on

Mechatronics 6(3)295ndash304 3 40

Frazzoli E Dahleh M and Feron E (2000) Trajectory tracking control design

for autonomous helicopters using a backstepping algorithm In American Control

Conference 2000 Proceedings of the 2000 volume 6 pages 4102ndash4107 IEEE 6

29

Hannaford B and Ryu J (2002) Time-domain passivity control of haptic interfaces

Robotics and Automation IEEE Transactions on 18(1)1ndash10 2

54

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 69: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 6973

Hirche S and Buss M (2004) Packet loss effects in passive telepresence systems

In Decision and Control 2004 CDC 43rd IEEE Conference on volume 4 pages

4010ndash4015 IEEE 2

Horn R and Johnson C (2005) Matrix analysis Cambridge university press 32

Hua M Hamel T Morin P and Samson C (2009) A control approach for thrust-

propelled underactuated vehicles and its application to vtol drones Automatic

Control IEEE Transactions on 54(8)1837ndash1853 6 26

Ji M and Egerstedt M (2005) Connectedness preserving distributed coordination

control over dynamic graphs In Proceedings of the American Control Conference

volume 1 page 93 36

Khalil H K (1996) Nonlinear systems Prentice hall Englewood Cliffs NJ 2nd ed

edition 7 17 32

Lam T Boschloo H Mulder M and Van Paassen M (2009) Artificial force field

for haptic feedback in UAV teleoperation Systems Man and Cybernetics Part A

Systems and Humans IEEE Transactions on 39(6)1316ndash1330 3

Lamb H (1895) Hydrodynamics Cambridge University Press Cambridge UK 1

Lawrence D (1993) Stability and transparency in bilateral teleoperation Robotics

and Automation IEEE Transactions on 9(5)624ndash637 2

Lee D Martinez-Palafox O and Spong M (2006a) Bilateral teleoperation of

a wheeled mobile robot over delayed communication network In Robotics and

Automation 2006 ICRA 2006 Proceedings 2006 IEEE International Conference

on pages 3298ndash3303 IEEE 2 34

Lee D and Spong M (2006a) Passive bilateral teleoperation with constant time

delay Robotics IEEE Transactions on 22(2)269ndash281 2 3

55

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 70: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7073

Lee D J (2007) Passivity-based switching control for stabilization of wheeled mobile

robots In Proc Robot Sci Sys pages 70 ndash71 12

Lee D J (2008) Semi-autonomous teleoperation of multiple wheeled mobile robots

over the internet In ASME Dynamic Systems amp Control Conference 5 6 14 15

18 19

Lee D J and Huang K (2008) Passive position feedback over packet-switching

communication network with varying-delay and packet-loss In Proc HAPTICS

pages 335ndash342 5

Lee D J and Huang K (2009a) Passive set-position modulation approach for

haptics with slow variable and asynchronous update In Proc HAPTICS pages

541ndash546 5

Lee D J and Huang K (2009b) Passive-set-position-modulation framework for

interactive robotic systems IEEE Transactions on Robotics 26(2)354 ndash369 2 5

16 25 38

Lee D J and Li P Y (2003) Passive bilateral feedforward control of linear

dynamically similar teleoperated manipulators IEEE Transactions on Robotics and Automation 19(3)443ndash456 14

Lee D J and Li P Y (2005) Passive bilateral control and tool dynamics rendering

for nonlinear mechanical teleoperators IEEE Transactions on Robotics 21(5)936ndash

951 14

Lee D J Martinez-Palafox O and Spong M W (2006b) Bilateral teleoperation

of a wheeled mobile robot over delayed communication network In Proc IEEE ICRA pages 3298ndash3303 2 3 4 14

Lee D J and Spong M W (2006b) Passive bilateral teleoperation with constant

time delay IEEE Transactions on Robotics 22(2)269ndash281 6

56

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 71: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7173

Lee S Sukhatme G Kim G and Park C (2002) Haptic control of a mobile

robot A user study In Proc IEEERSJ IROS pages 2867ndash2874 3 4 14

Leung G Francis B and Apkarian J (1995) Bilateral controller for teleoperators

with time delay via micro-synthesis Robotics and Automation IEEE Transactions on

11(1)105ndash116 2

Lim J Ko J and Lee J (2003) Internet-based teleoperation of a mobile robot

with force-reflection In Proc IEEE CCA pages 680ndash685 1 3

Mahony R and Hamel T (2004) Robust trajectory tracking for a scale model

autonomous helicopter International Journal of Robust and Nonlinear Control

14(12)1035ndash1059 6

Niemeyer G and Slotine J (1991) Stable adaptive teleoperation Oceanic

Engineering IEEE Journal of 16(1)152ndash162 2

Niemeyer G and Slotine J (1998) Towards force-reflecting teleoperation over the

internet In Robotics and Automation 1998 Proceedings 1998 IEEE International

Conference on volume 3 pages 1909ndash1915 IEEE 2

Niemeyer G and Slotine J (2004) Telemanipulation with time delays International

Journal of Robotics Research 23(9)873ndash890 6

Nourbakhsh I Sycara K Koes M Yong M Lewis M and Burion S (2005)

Human-robot teaming for search and rescue IEEE Pervasive Computing pages

72ndash78 1

Nuno E Basanez L Ortega R and Spong M (2009a) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 2

57

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 72: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7273

Nuno E Basanez L Ortega R and Spong M (2009b) Position tracking for

non-linear teleoperators with variable time delay The International Journal of

Robotics Research 28(7)895 5

Peshkin M and Colgate J (1999) Cobots Industrial Robot 26(5)33ndash34 1

Rodrıguez-Seda E Troy J Erignac C Murray P Stipanovic D and Spong M

(2010) Bilateral teleoperation of multiple mobile agents coordinated motion and

collision avoidance Control Systems Technology IEEE Transactions on 18(4)984ndash

992 3

Salcudean S Zhu M Zhu W and Hashtrudi-Zaad K (2000) Transparent

bilateral teleoperation under position and rate control The International Journal of Robotics Research 19(12)1185 2

Schenker P Huntsberger T Pirjanian P Baumgartner E and Tunstel E (2003)

Planetary rover developments supporting mars exploration sample return and

future human-robotic colonization Autonomous Robots 14(2)103ndash126 1

Slawinski E Mut V and Postigo J (2007) Teleoperation of mobile robots with

time-varying delay IEEE Transactions on Robotics 23(5)1071ndash1082 3

Stramigioli S Mahony R and Corke P (2010) A novel approach to haptic tele-

operation of aerial robot vehicles In Robotics and Automation (ICRA) 2010 IEEE

International Conference on pages 5302ndash5308 IEEE 3

Valavanis K (2007) Advances in unmanned aerial vehicles state of the art and the

road to autonomy Springer Verlag 1

Zuo Z and Lee D J (2010) Haptic Tele-Driving of a Wheeled Mobile Robot over

the Internet a PSPM Approach In Proc IEEE CDC pages 3614ndash3619 1

Zuo Z and Lee D J (2011) Backstepping Control of Quadrotor-type UAVs

Trajectory Tracking and Teleoperation over the Internet In To submit 1

58

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation

Page 73: Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

7182019 Haptic Tele-operation of Wheeled Mobile Robot and Unmanned Aerial

httpslidepdfcomreaderfullhaptic-tele-operation-of-wheeled-mobile-robot-and-unmanned-aerial 7373

Vita

Zhiyuan Zuo was born in 1987 Shandong China He received his bachelorrsquos degree in

2009 Harbin Institute of Technology in Automotive Engineering He joined robotics

laboratory since sophomore year and worked hard there trying to be a good engineer

After college he came to US for further study in Robotics and control In graduationstudy he worked as a research assistant in INRoL UTK and also as a teaching

assistant in Mechatronics lab class He would begin his career as an engineer after

graduation