Post on 03-May-2018
Haptic Teleoperation for Robot-Assisted
Surgery
Chao LIU
CR1 Research Scientist National Center for Scientific Research (CNRS)
LIRMM, UMR 5506 Montpellier, France
liu@lirmm.fr
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
1/45
INTRODUCTION TO TELEOPERATION
| Teleoperation: “ is the extension of a person’s sensing and manipulation
capability to a remote location” - Sheridan 1989
2/45
INTRODUCTION TO TELEOPERATION
| Brief history y Early period
fire-tongs Prod, spear, etc
Polygraph: produces a copy of a piece of writing (Thomas Jefferson ~1805)
3/45
INTRODUCTION TO TELEOPERATION
y Teleoperation systems were developed in the mid-1940s to create capabilities for handling highly radioactive material
4/45
INTRODUCTION TO TELEOPERATION
y First modern master-slave teleoperation system (electrically servoed instead of mechanically) - Argonne National Laboratory, US under Ray Goertz in 1954
master slave
communication 5/45
INTRODUCTION TO TELEOPERATION
y MASCOT – (MAnipulatore Servo COntrollato Transistorizzato) Carlo Mancini et al, 1958
IX Congress of Electronics Exhibition in Roma, 1962
6/45
INTRODUCTION TO TELEOPERATION
| Teleoperation: “ is the extension of a person’s sensing and manipulation
capability to a remote location” - Sheridan 1989
Human operator (master) acts as a supervisor, communicating to a computer information relative to a limited task (goals, constrains, plans…) and getting back information (accomplishments, difficulties, sensory data…)
Teleoperator (slave) is a machine that enables a human operator to move about, sense and mechanically manipulate objects at a distance
7/45
INTRODUCTION TO TELEOPERATION
| Current teleoperation applications: Industry; Space, deap sea, planetary exploration; Hazard material handling; Micro/Nano manipulation etc.
8/45
INTRODUCTION TO TELEOPERATION
| Medical and Surgical Teleoperation
The da Vinci® Surgical System Till 2013, there are 2,585 da Vinci systems installed in 2,025
hospitals around the world
9/45
INTRODUCTION TO TELEOPERATION
| Human operator
| Teleoperator
10/45
INTRODUCTION TO TELEOPERATION
Da Vinci® Telesurgery System | Advantages
y Dedicated for Minimally Invasive Surgery (MIS)
¾ Remove surgeon’s hand tremor ¾ Improved 3D visual monitoring ¾ Increased range of motion
¾ Enhanced dexterity ¾ Greater operation precision ¾ Wide instrument choices
VS
11/45
Da Vinci® Telesurgery System | Disadvantages
y High cost (~ 2M USD) y Steep learning curve y No force feedback to the surgeon Æ lack the tactile or haptic sensation (the cability to “feel” the tissues being operated on)
INTRODUCTION TO TELEOPERATION
the natural feeling of operating is lost (negative effects on operation accuracy and safety)
Unilateral Control 12/45
INTRODUCTION TO TELEOPERATION
| Challenges for teleoperation in robot-assisted surgery? y Provide force feedback to the surgeon during operation
Bilateral Control Unilateral Control
Input command (desired output) is not influenced by
any physical variable present at the output
Output state affects the input and vice-versa
13/45
INTRODUCTION TO TELEOPERATION
| Challenges for teleoperation in robot-assisted surgery? y Deal with Time delay
Endoscopic Capsule Robots
Modular Robots
Å wireless communication
14/45
HAPTIC TELEOPERATION
How to design a haptic/bilateral teleoperation system
for robot-assisted surgery
Æ Stability
Æ Transparency
To maintain stability of the closed-loop system irrespective of the behavior of the operator or the environment -> safety!
To provide a faithful transmission of signals (positions, velocities, forces) between master and slave to couple the operator as closely as possible to the remote task -> telepresence
15/45
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
HAPTIC TELEOPERATION FUNDAMENTAL
Although bilateral systems can be modeled with traditional block diagram or signal-flow-graph techniques, more insight is obtained with methods drawn from the network theory which share this bilateral property
16/45
HAPTIC TELEOPERATION FUNDAMENTAL
| Basic Concepts: y Lumped Parameter Elements Physical entities who’s energy (storage elements) or power (dissipative
elements) is defined by a scalar. Examples: mass resistor
y Network A system described by lumped parameter elements connected in series and
parallel. Each element consists of one constitutive relation. Example: An electric circuit consisting of R, L, C in arbitrary connections.
y Port A location where energy can move into or out of a network. An effort and
flow, are defined for each port. Examples: Electric wall socket. Contact point between human and haptic
interface.
17/45
HAPTIC TELEOPERATION FUNDAMENTAL
y Effort and Flow The flow of energy between two elements is characterized by two
variables called effort and flow , of which the product is power. y Impedance and Admittance Effort and flow variables are linked together by the behavior of
systems at their ports, with relationship represented as:
Impedance Admittance
18/45
HAPTIC TELEOPERATION FUNDAMENTAL
Port: location at which energy can move into and out of the system y One-port network A system which has a single port is called a one-port system or simply
a one-port
Convention: 1) Choose the sign of flow so that it is positive going into the port.
2) The effort of interest at the port is that being exerted on the port, not the effort exerted by the port on the environment.
3) Choose the sign of effort so that when energy flows into the port Æ effort has the same sign as flow
effort and flow define positive or negative power going into the network
19/45
HAPTIC TELEOPERATION FUNDAMENTAL
y Two-port network A system with two ports Two-port has separate effort and flow variables defined for each
port, and a separate coordinate system (sign convention) for each port.
Teleoperation network block diagram 20/45
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
1/45
HAPTIC TELEOPERATION DESIGN
Passivity-Based Teleoperation
- Passivity theory is an input–output property of dynamical systems that has its origins in network theory and is concerned primarily with the exchange of energy between interconnected systems
- System passivity constitutes a sufficient condition for the stability of the system coupled to a passive environment and a bounded input energy operator, and represent a necessary condition for stability with any remote environment
21/45
HAPTIC TELEOPERATION DESIGN
y Passivity definition - system input x - system output y - “power” P entering system as A system is said to be passive if: where E is lower bounded “energy storage” function, is non-
negative “power dissipation” function. the power is either stored or dissipated total energy supplied by the system is lower bounded
May not correspond to actual physical quantities
22/45
HAPTIC TELEOPERATION DESIGN
y Important practical feature of passivity formulation: The combination of passive systems connected in either a serial or
feedback/parallel configuration is still passive
network elements could be studied separately
23/45
| Stability Robustness and Performance Evaluation Tools
operator force on the master slave force on the environment master velocity slave velocity master impedance slave impedance external force generated by the operator external force generated by environment
HAPTIC TELEOPERATION DESIGN
24/45
| Stability Robustness and Performance Evaluation Tools y Depending on the choice of the network input and output
variables y and u (effort and flow), their relationship is described in terms of an immittance matrix P such that y = P×u:
Impedance Matrix Hybrid Matrix
HAPTIC TELEOPERATION DESIGN
Admittance Matrix
Alternate Hybrid Matrix
»¼
º«¬
ª»¼
º«¬
ª »
¼
º«¬
ª
e
h
e
h
VV
zzzz
FF
2221
1211»¼
º«¬
ª»¼
º«¬
ª »
¼
º«¬
ª
e
h
e
h
FF
yyyy
VV
2221
1211
»¼
º«¬
ª»¼
º«¬
ª »
¼
º«¬
ª� e
h
e
h
FV
hhhh
VF
2221
1211»¼
º«¬
ª�»
¼
º«¬
ª »
¼
º«¬
ª
e
h
e
h
VF
gggg
FV
2221
1211
eehhT VFVFuy �
25/45
HAPTIC TELEOPERATION DESIGN
| Stability Robustness and Performance Evaluation Tools y Hybrid matrix parameter interpretation
»¼
º«¬
ª»¼
º«¬
ª »
¼
º«¬
ª� e
h
e
h
FV
hhhh
VF
2221
1211
022
021
012
011
h
e
e
e
Ve
e
Fh
e
Ve
h
Fh
h
FVh
VVh
FFh
VFh Free motion input impedance
Force scaling
Velocity scaling
Output admittance with clamped input
26/45
HAPTIC TELEOPERATION DESIGN
| Stability Robustness and Performance Evaluation Tools y Llewellyn’s absolute stability condition An LTI two-port network is absolutely stable if and only if for any P
matrix: 1. p11 and p22 are positive real (no poles in the open right-half-plane, any
imaginary poles are simple and have real and positive residues) 2. Either of the following 2 sets of inequalities is satisfied:
or
Network stability parameter
27/45
HAPTIC TELEOPERATION DESIGN
| Stability Robustness and Performance Evaluation Tools y Transparency Transparency can be described quantitatively as a match between
the environment impedance and the impedance transmitted to the operator:
»¼
º«¬
ª�
0110
idealH
A perfect transparent system is marginally absolutely stable !
28/45
HAPTIC TELEOPERATION DESIGN
| Architectures of Bilateral Teleoperation
How to realize the bilateral teleoperation system?
Teleoperation network block diagram
29/45
HAPTIC TELEOPERATION DESIGN
| Architectures of Bilateral Teleoperation
General 4-channel Architecture
30/45
HAPTIC TELEOPERATION DESIGN
| Architectures of Bilateral Teleoperation y Two channel architectures
Position-Position architecture
C2=C3=0
31/45
HAPTIC TELEOPERATION DESIGN
| Architectures of Bilateral Teleoperation y Two channel architectures
Position-Force architecture
C3=C4=0
32/45
HAPTIC TELEOPERATION DESIGN
| Architectures of Bilateral Teleoperation y Two channel architectures
33/45
HAPTIC TELEOPERATION DESIGN
| Bilateral Teleoperation Design Guidance y In practice, perfect transparency and robust stability (passivity) are
conflicting design goals for bilateral teleoperation systems
y With specific control architecture and system properties, both stability and transparency are compromised and their analysis becomes involved
Æ use the stability robustness and performance evaluation tools to numerically evaluate
34/45
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
HAPTIC TELEOPERATION DESIGN
Time delay
What effects? Æ Destroy system passivity! Æ Potentially unstable
35/45
HAPTIC TELEOPERATION DESIGN
Time delay
| Analysis through Scattering theory y Scattering operator : For an n-port with effort F and flow v, the scattering operator S
is defined by:
using Hybrid matrix for 2 port system
y Theorem: A system is passive if and only if the norm of its scattering
operator S is less than or equal to one:
»¼
º«¬
ª»¼
º«¬
ª »
¼
º«¬
ª� e
h
e
h
FV
hhhh
VF
2221
1211
36/45
HAPTIC TELEOPERATION DESIGN
Time delay
| Analysis through Scattering theory y With existence of time delay:
Direct transmission of force and velocity
signal with time delay is not passive!
37/45
HAPTIC TELEOPERATION DESIGN
| Wave Variable Design Method y Instead of transmitting force and velocity signals, wave variables
(input waves u and output waves v) are transmitted
38/45
HAPTIC TELEOPERATION DESIGN
| Wave Variable Design Method
Recall power and eehhT VFVFuyP �
Passive for arbitrary time delay!
39/45
HAPTIC TELEOPERATION DESIGN
| Wave Variable Design Method y Transparency
¾ since wave variable is a excessively conservative approach to system passivity, the transparency of the standard wave-variable-based system is degraded.
¾ trade-off between stability and transparency is again inherent in the teleoperation system
¾ the fundamental tuning parameter in a wave-variable based controller is the wave impedance b
40/45
HAPTIC TELEOPERATION DESIGN
| Wave Variable Design Method y Wave reflection waves are reflected at points (junctions and terminations) where
the wave carrier impedance changes (master and slave sites), which corrupt the useful information flow and cause oscillatory behavior
41/45
HAPTIC TELEOPERATION DESIGN
| Wave Variable Design Method y Impedance matching for wave reflection the impedance of the wave transmission can be matched to the
remaining system by including additional termination elements.
42/45
HAPTIC TELEOPERATION DESIGN
| Main Problem transparency and passivity are conflicting design goals
Transparency Passivity
Trade-off
Safety Maximize transparency Scaling factor Force/Position tracking etc.
Robotic surgery requirements
43/45
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
NEW CHALLENGES FOR HAPTIC ROBOTIC SURGERY
| New Challenges/Open Questions for Haptic Robotic Surgery ¾ Better modeling of the modules (tissue model)
¾ Active environment (beating heart, etc)
¾ Sensor (biocompatibility, sterizability, size, cost)
¾ Asymmetrices of sensor/actuator
¾ Wireless telesurgery (communication & power transfer)
¾ Cognition in telesurgery (virtual fixtures)
¾ Semi-autonomous telesurgery
¾ New slave surgical robot (concentric tube robot)
¾ … …
44/45
OUTLINE
| Introduction to teleoperation
| Haptic/Bilateral teleoperation fundamentals
| Design of haptic teleoperation for robot-assisted surgery y Passivity and transparency
y Time delay
| New challenges & open issues
| Conclusion
CONCLUSIONS
| Conclusions y Brief history of teleoperation
y Why haptic teleoperation is necessary for robot-assisted surgery
y Fundamentals of haptic teleoperation
y Passivity and transparency of haptic teleoperation
y Stability Robustness and Performance Evaluation Tools
y Time-delayed haptic teleoperation
y New challenges & open issues for haptic robot-assisted surgery
45/45
HAPTIC TELEOPERATION FOR ROBOT-ASSISTED SURGERY
Thank You!
Chao LIU
LIRMM – CNRS liu@lirmm.fr
www.lirmm.fr/~liu