Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

EUROBOT Conference 2009

CONTROL METHODOLOGIES FOR ENDOSCOPE NAVIGATION IN ROBOTIZED LAPAROSCOPIC

SURGERY

Enrique Bauzano Núñez, Dr. Víctor Muñoz MartínezDra. Isabel García Morales, Belén Estebanez Campos

Dpto. Ingeniería de Sistemas y Automáticahttp://www.isa.uma.es

Universidad de Málaga (Spain)

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

OUTLINE

I. Introduction

II. Laparoscopic Navigation Problem

III.Force Interaction Model on Active Wrists

IV.Control Strategy

V. Implantation and Experiments

VI.Conclusions and Future Works

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

I. INTRODUCTION

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

I. Introduction

Laparoscopic Surgery: “Accessing to the abdominal cavity with especial long instruments through small incisions.”

LAPAROSCOPIC SURGERY

Advantages Lessen recovery time

Limit post-operative complications

Lower scars

Constraints Movement limitations

Loss of touch and 3D perception

Hand-eye coordination problems

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

I. Introduction

Proposal: “Solve the motion problem for a laparoscopic surgery robot assistant.”

GLOBE-SHAPED MOVEMENTS

{I} Fulcrum point Orientation angle Altitude angle External distance

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ROBOTIC WRISTS COMPARATIVE

• Null forces on abdomen• Precision depends on fulcrum accuracy

• Null forces on abdomen• Special instruments needed• Previous calibration• Voluminous mechanisms

• No additional mechanisms on wrist• No special instruments needed• Low volume• May apply forces on abdomen

I. Introduction

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

II. LAPAROSCOPIC NAVIGATIONPROBLEM

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

PASSIVE WRISTS MECHANISMII. Laparoscopic Navigation Problem

External supply is not necessary.

Controller is integrated on the arm itself.

Adaptable to operating room requirements.

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

NAVIGATION PROBLEM ON PASSIVE WRISTS

XI

ZI

C

C

e

ANavigation Problem: Move the endoscope from A to B an altitude angle .

I0

I0 Initial fulcrum (desired)

C Estimated fulcrum Orientation error

e External distance error

e Altitude error

II. Laparoscopic Navigation Problem

B

ʹ

Bʹe

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRISTS MECHANISMII. Laparoscopic Navigation Problem

Force Sensor

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

NAVIGATION PROBLEM ON ACTIVE WRISTS

I

XI

ZI

CC

e

AB

es

Navigation Problem: Move the endoscope from A to B an altitude angle .

I0

I0 Initial fulcrum (desired)

I Actual fulcrum

C Estimated fulcrum

e External distance error

es Separation error

II. Laparoscopic Navigation Problem

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

III. FORCE INTERACTION MODEL ON ACTIVE WRISTS

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ENDOSCOPE-ABDOMEN INTERACTION MODEL

XI

ZI

Fr

III. Force Interaction Model on Active Wrists

rs FFgF m

A

Cmg

I0

Fs

I

B

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

FORCES MODEL

Stiffness Coefficient

0 0.5 1 1.5-3

-2

-1

0

1

2

3

4

Time (seconds)

Ela

stic

ity

(N/m

m)

Real Estimated

-20

0

20

00.5

11.5

2

0

2

4

6

8

10

Altitude (degrees)Time (seconds)

Fric

tion

F r (New

ton)

ss eF NFr

III. Force Interaction Model on Active Wrists

Friction Coefficient

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

IV. CONTROL STRATEGY

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

PASSIVE WRISTS: CONTROL SCHEMEIV. Control Strategy

Robot & Endoscope

Trajectory Generator Planner

Control PI

Geometrical Estimator

Actual Endoscope Location

Required Spherical Trajectory

Endoscope Desired Location

+-

Estimated External Distance

Real External Distance

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRISTS: NAVIGATION STRATEGY

1. Rotate around estimated fulcrumFulcrum displacement over the abdomen

2. Passive Wrist BehaviorEndoscope rotation over the wrist to reduce fulcrum displacement

3. Altitude angle correctionRecover desired altitude angle

I

L1L3

XI

ZI

C

IV. Control Strategy

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRISTS: PASSIVE WRIST BEHAVIOR (I)IV. Control Strategy

Trajectory Generator Planner

Robot & Endoscope

Abdomen Interaction

Force Sensor

Separation estimator

External Distance estimator

KFriction + Weight

Filtering

Cor

rect

ion

Ang

le

Planned Movement

Followed Altitude

Abdomen Force Fs Force F

Planned Altitude p

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRISTS: PASSIVE WRIST BEHAVIOR (II)IV. Control Strategy

Planner– Calculates the required trajectory with a first order system behavior– Acts faster than correction angle loop

Robot: Performs the controlled movement

Endoscope-Abdomen Interaction: Generates abdominal forces

Force Sensor: Measurement of forces

Separation Estimator– Depends on the stiffness estimation– Its value remains the same if Fs < Fthreshold

External Distance Estimator– Result of Force-Torque balance– Its value remains the same if Fs < Fthreshold

rT

sT

s LekLekL )1()()1(

e

ssFe

Ce

sρ

s

FuM

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRISTS: SPHERICAL CONTROL LOOPIV. Control Strategy

Passive Wrist Behavior Robot

Followed Altitude

Control PI

+-

Desired Altitude d

Planned Altitude p

Controlled Altitude c

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

V. IMPLANTATION AND EXPERIMENTS

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

PASSIVE WRIST IMPLANTATION (I)V. Implantation and Experiments

Surgeon Voice Recognizer Control Software

Controller

ERM Robot

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRIST IMPLANTATION (II)V. Implantation and Experiments

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRIST IMPLANTATION (I)V. Implantation and Experiments

Movement Command

Virtual-PA10 simulator (Matlab)

Matlab-C++ DLL Library

linker

C++ Library PA-10 ControllerPA-10 Robot

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE WRIST IMPLANTATION (II)V. Implantation and Experiments

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

EXPERIMENTAL RESULTSV. Implantation and Experiments

0 1 2 3 4 5 6-49

-48

-47

-46

-45

-44

-43

-42

-41

-40

Time (seconds)

Alti

tude

(d

egre

es)

Desired Followed

0 1 2 3 4 5 6 7 8 9-49

-48

-47

-46

-45

-44

-43

-42

-41

Altit

ude

angl

e (d

egre

es)

Time (seconds)

Actual trajectoryDesired trajectoryRequired trajectory

Passive Wrists Active Wrists

0 1 2 3 4 5 6 7 8 993.5

93.6

93.7

93.8

93.9

94

94.1

94.2

94.3

94.4

94.5

Orie

ntat

ion

angl

e (d

egre

es)

Time (seconds)

Actual trajectoryDesired trajectoryRequired trajectory

0 2 4 6 8 10266

268

270

272

274

276

278

Time (seconds)

Exte

rnal

Dist

ance

e (

mm

)

EstimatedReal

0 2 4 6 8 100

0.5

1

1.5

2

2.5

3

3.5

Time (seconds)

Sepa

ratio

n Er

ror e

S (mm

)

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

VI. CONCLUSIONS AND FUTURE WORKS

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

ACTIVE VS. PASSIVE WRISTS

Almost no fluctuations to reach steady state on altitude angle

No fluctuations on orientation angle

May be used to apply efforts with surgery tools

LAST IMPROVEMENTS ON ACTIVE WRISTS

System does not depend on friction force

Stiffness can be now measured on-line, not model needed

New control methodology for stability analysis

V. Conclusions and Future Works

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

FUTURE WORKS

Active tasks during an intervention

Development of semi-autonomous robotic surgeon assistants

Like remote-centered wrists, useful on tele-surgery fields

V. Conclusions and Future Works

Syst

em E

ngin

eerin

g an

d A

utom

atio

n D

epar

tmen

t

Enrique Bauzano Nuñez ebauzano@uma.es

Control Methods for Endoscope Navigation in Robotized Lap. Surgery

THANKS FOR YOUR ATTENTION!!!