EMBEDDED OPTICAL SENSING FOR ROBOTS IN EXTREME …ry064nm3195/YLP_thesis-augmented.pdffor certain...

EMBEDDED OPTICAL SENSING FOR ROBOTS IN EXTREME

ENVIRONMENTS

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF MECHANICAL ENGINEERING

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Yong-Lae Park

March 2010

http://creativecommons.org/licenses/by/3.0/us/

This dissertation is online at: http://purl.stanford.edu/ry064nm3195

© 2010 by Yong-Lae Park. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

This work is licensed under a Creative Commons Attribution-3.0 United States License.

ii



http://purl.stanford.edu/ry064nm3195

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Mark Cutkosky, Primary Adviser


Oussama Khatib


Richard Black

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

Abstract

Force sensing is an essential requirement for dexterous robot manipulation. Metal

or semiconductor strain gages are commonly used for measuring forces. However,

for certain uses in extreme environments, such as extra-vehicular activities in space

and magnetic resonance imaging (MRI)–guided robotic surgeries, optical fiber Bragg

grating (FBG) sensors have compelling advantages: they are immune to electromag-

netic noise, physically robust (especially when embedded in solid parts), and able

to resolve very small strains. In addition, with optical multiplexing, many sensors

can be located along a single fiber and interrogated in parallel.

This thesis first describes composite robot end-effectors that incorporate optical

fibers for accurate force sensing and control and for estimating contact locations.

The overall design is inspired by biological mechanoreceptors, such as slit sensillae

or campaniform sensillae, in arthropod exoskeletons that allow them to sense con-

tacts and loads on their limbs by detecting strains caused by structural deformation.

A new fabrication process is presented to create multi-material reinforced robotic

structures with embedded fibers. The results of experiments are presented for char-

acterizing the sensors and controlling contact forces in a closed loop system involving

a large industrial robot and a two fingered dexterous hand. The proposed exoskele-

ton finger structure was able to detect less than 0.02 N of contact force changes

and to measure less than 0.15 N of contact forces practically. A brief description on

the optical interrogation method, used for measuring multiple sensors along a single

fiber at kHz rates needed for closed-loop force control, is also provided.

iv

Following the successful creation of force sensing robot fingers in a large-scale

(120 mm long) prototype, the finger structure was miniaturized to a human finger-

tip scale (15 mm long) for robots designed for human interactions in space. For

miniaturization, a bend-insensitive optical fiber was selected to be embedded in a

small fingertip. The small fingertip was able to measure 3-axial forces with practical

force measurement resolutions of 0.05 N for forces applied transverse to the finger

and 0.16 N for forces applied axially to the fingertip.

As an extension of the application of FBG sensors to robotic devices used in ex-

treme environments, a magnetic resonance imaging (MRI)–compatible biopsy needle

is instrumented with FBGs for measuring bending deflections as it is inserted into

tissues. During procedures such as diagnostic biopsies and localized treatments, it

is useful to track any tool deviation from the planned trajectory to minimize posi-

tioning error and procedural complications. The goal is to display tool deflections in

real-time, with greater bandwidth and accuracy than when viewing the tool in MR

images. A standard 18 ga (≈ 1 mm diameter) × 15 cm inner needle is prepared us-

ing a custom-designed fixture, and 350 µm deep grooves are created along its length.

Optical fibers are embedded in the grooves. Two sets of sensors, located at different

points along the needle, provide a measurement of an estimate of the bent profile,

as well as temperature compensation. After calibration, the measured tip position

was accurate to within 0.11 mm. Tests of the needle in a canine prostate showed

that it produced no adverse imaging artifacts when used with the MR scanner and

no sensor signal degradation from the strong magnetic field.

v

Acknowledgements

Foremost, I would like to express my sincerest gratitude to my advisor, Professor

Mark R. Cutkosky. His insight and knowledge in the field of robotics together with

his hearty support made my whole program of study really fruitful. He was always

kind, patient and understanding whether I was making good progress or not. He

encouraged me to think about the meaning of my research instead of just showing

what I needed to achieve. He was also fully understanding of my situation when I

needed to take my time off from study and research to take care of my mother. I

could never have come so far and accomplished this much without his support and

advice.

I am also grateful to Professor Oussama Khatib for his suggestions regarding the

writing of this manuscript. I would like to thank Dr. Richard J. Black of Intelligent

Fiber Optic Systems Corporation (IFOS) for assisting me with his expertise in the

field of fiber optics. He was always happy to provide any support and advice on my

work related to fiber optics. He also gave valuable comments on the writing of this

manuscript. I would also like to thank Professor Bruce Daniel who provided insight

and knowledge in the field of medical robotics and MRI-guided interventions. I also

wish to thank Professor Ken Waldron for his role as my defense committee member.

Thanks are due to IFOS members, especially Dr. Behzad Moslehi, CEO/CTO,

for his initiation and support of the research and fruitful discussion, and Levy Oblea

for technical support in fiber optics.

vi

Thanks are also given to my colleagues and fellow graduate students at Stan-

ford, especially Sangbae Kim, Santhi Elayaperumal, Seok Chang Ryu, Karlin Bark,

Li Jiang, Daniel Santos, William Provancher, Trey McClung, Jason Wheeler, Alan

Asbeck, Aaron Parness, Pete Shull, Mihye Shin, Sanjay Dastoor, Salomon Trujillo,

Barret Heynman, and Daniel Aukes for all the thoughtful discussions, mutual teach-

ing and moral support throughout this experience. Special thanks go to Seok Chang

and Santhi for their help with Sections 3.5, 5.4, and 5.5.

I would like to express deep appreciation to my parents and my sisters, Hye-Sung

and Sun-Kyoung, for their unconditional love, care and support. I am especially

grateful to my mother who overcame her long and severe illness.

Finally, I am truly grateful for the enthusiastic support of my fiancee, Su-Yeon.

Financial support of this work was provided by the National Aeronautics and

Space Administration (NASA) under SBIR NNJ06JA36C, and the U.S. Army Med-

ical Research Acquisition Activity (USAMRAA) under STTR W81XWH8175M677.

vii

Contents

Abstract iv

Acknowledgements vi

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background Information 5

2.1 Fiber Bragg Grating (FBG) Sensors . . . . . . . . . . . . . . . . . . . 5

2.2 Shape Deposition Manufacturing (SDM) . . . . . . . . . . . . . . . . 8

3 Exoskeletal Force Sensing End-Effectors 10

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2 Design Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.2.1 Exoskeleton Structure . . . . . . . . . . . . . . . . . . . . . . 13

3.2.2 Creep Prevention and Thermal Shielding . . . . . . . . . . . . 15

3.2.3 Strain Sensor Configuration . . . . . . . . . . . . . . . . . . . 16

3.2.4 Temperature Compensation . . . . . . . . . . . . . . . . . . . 16

3.3 Prototype Fabrication . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3.1 Material Selection . . . . . . . . . . . . . . . . . . . . . . . . . 18

viii

3.3.2 SDM Fabrication Process . . . . . . . . . . . . . . . . . . . . 20

3.4 Static and Dynamic Characterization . . . . . . . . . . . . . . . . . . 20

3.4.1 Static Force Sensing . . . . . . . . . . . . . . . . . . . . . . . 22

3.4.2 Modes of Vibration . . . . . . . . . . . . . . . . . . . . . . . . 23

3.4.3 Hysteresis Analysis . . . . . . . . . . . . . . . . . . . . . . . . 25

3.4.4 Temperature Compensation . . . . . . . . . . . . . . . . . . . 27

3.4.5 Contact Force Localization . . . . . . . . . . . . . . . . . . . . 27

3.5 Force Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.6 Contact force control . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.6.1 Results of Experiments . . . . . . . . . . . . . . . . . . . . . . 36

3.7 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . 38

4 Miniaturized Force Sensing Fingers 40

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.2 Design Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.2.1 Bend-insenstive Optical Fibers . . . . . . . . . . . . . . . . . . 42

4.2.2 Structure Design . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.2.3 Sensor Configuration . . . . . . . . . . . . . . . . . . . . . . . 44

4.2.4 Creep Prevention . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.3 SDM Fabrication Process . . . . . . . . . . . . . . . . . . . . . . . . . 45

4.4 Force Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.5 Improvement of z-axis Force Sensitivity . . . . . . . . . . . . . . . . . 48

4.5.1 Design Modification . . . . . . . . . . . . . . . . . . . . . . . 50

4.5.2 SDM Fabrication Process . . . . . . . . . . . . . . . . . . . . 50

4.5.3 Force Calibration . . . . . . . . . . . . . . . . . . . . . . . . . 52


5 MRI-Compatible Shape Sensing Needle 57

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5.2 System Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

ix

5.3 Prototype Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.3.1 Sensor Configuration . . . . . . . . . . . . . . . . . . . . . . . 62

5.3.2 Inner Stylet Design . . . . . . . . . . . . . . . . . . . . . . . . 71

5.4 Needle Fabrication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.5 Sensor Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.5.1 Wavelength Shift vs. Curvature . . . . . . . . . . . . . . . . . 73

5.5.2 Wavelength Shift vs. Deflection . . . . . . . . . . . . . . . . . 79

5.6 System Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.7 Preliminary MRI Scanner Tests . . . . . . . . . . . . . . . . . . . . . 84

5.8 Preliminary In-vivo Testing . . . . . . . . . . . . . . . . . . . . . . . 84


6 Conclusions 88

6.1 Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Bibliography 93

x

List of Tables

3.1 Material properties of polyurethane candidates. . . . . . . . . . . . . 19

3.2 Parameters of embedded FBG sensors . . . . . . . . . . . . . . . . . . 22

5.1 Boundary conditions used for needle deflection estimation based on

beam theory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.2 Deflection error comparison for different deflection ranges using cur-

vature calibration method. . . . . . . . . . . . . . . . . . . . . . . . . 77

5.3 Deflection error comparisons for different deflection ranges with direct

deflection estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

xi

List of Figures

2.1 Fiber Bragg grating structure with spectral response . . . . . . . . . 6

2.2 SDM fabrication process with a continuous, alternating machining

and deposition cycles [4] . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 (A) Prototype dimensions. (B) FBG embedded force sensing finger

prototypes integrated with two fingered hand and industrial robot. . . 13

3.2 Completed finger prototype. The prototype can be divided into three

parts: fingertip, shell, and joint. . . . . . . . . . . . . . . . . . . . . . 14

3.3 Design details of the finger prototype with cross-sectional views (S1−S4: strain sensors, S5: temperature compensation sensor). See Table

3.2 for sensor parameters. . . . . . . . . . . . . . . . . . . . . . . . . 15

3.4 Finite element models showing strain concentrations on the first rib

closest to the fixed joint. (A) Point load is applied to the fingertip.

(B) Point load is applied to the middle of the shell structure. . . . . . 17

3.5 Prototype comparison with different materials. . . . . . . . . . . . . . 19

xii

3.6 Modified SDM fabrication process: [Step 1] Shell fabrication (a) Pre-

pare a silicone rubber inner mold and place optical fibers with FBG

sensors. (b) Wrap the inner mold with copper mesh. (c) Enclose

the inner mold and copper mesh with a wax outer mold and pour

liquid polyurethane. (d) Remove the inner and outer molds when the

polyurethane cures. [Step 2] Fingertip fabrication (a) Prepare inner

and outer molds and place copper mesh. (b) Cast liquid polyurethane.

(c) Place the cured shell into the uncured polyurethane. (d) Remove

the molds when the polyurethane cures. [Step 3] Joint fabrication (a)

Prepare an outer mold and place a temperature compensation sensor

structure. (b) Place the cured shell and fingertip into the uncured

polyurethane. (c) Remove the outer mold when polyurethane cures. . 21

3.7 Wax and silicone rubber molds and copper mesh used in modified

SDM fabrication process. . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.8 Static force response results. (A) Shell force response. (B) fingertip

force response. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.9 (A) Impulse response of the finger prototype. (B) Fast Fourier trans-

form of impulse response. . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.10 Modes of vibration of the finger prototype using finite element anal-

ysis. Modes 1 and 2 are the dominant modes, representing bending

about x and y axes, respectively. . . . . . . . . . . . . . . . . . . . . 27

3.11 The effect of applying a steady load for several seconds and suddenly

removing it from the polymer fingertip. . . . . . . . . . . . . . . . . . 28

3.12 Detailed views of creep under steady loading (A) and of the hysteresis

associated with sudden unloading (B). . . . . . . . . . . . . . . . . . 28

3.13 Test result showing partial temperature compensation provided by

the central sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.14 2D simplified shell structure and deformations of finger prototype. . . 30

xiii

3.15 Strain ratio plot of sensor A to B (εA/εB) with error estimates for

several locations of force application along the length of the finger. . . 31

3.16 (A) Top view of the prototype showing embedded sensors and force

application. (B) Plot of sensor signal outputs. . . . . . . . . . . . . . 32

3.17 Hardware system architecture. . . . . . . . . . . . . . . . . . . . . . . 33

3.18 Position based force control system. F and Fr are contact force and

user-specified force setpoint. X, Xc, Xf , and Xr are respectively ac-

tual position, commanded position, position perturbation computed

by the force controller, and reference position of the end-effector. . . . 35

3.19 Experimental results of force setpoint tracking. (A) Adept robot

motion. (B) Joint angle change of Dexter manipulator. (C) Force

data from load cell and FBG embedded robot finger prototype. Robot

starts force control as soon as it makes a contact with the object at

t1. Robot starts to retreat at t2. Robot breaks contact at t3. . . . . . 37

3.20 Experimental results of force control during manipulation tasks (A)

Grasp force measured by a finger with FBG sensors (B) Acceleration

plotted along with magnitude of combined (x, y, and z) acceleration

of the robot. Periods a, b, e, and f are for translation motions. Periods

c and d are for rotation motions. Every task motion is followed by a

waiting period before starting next motions. . . . . . . . . . . . . . . 39

4.1 Miniaturized polyurethane finger prototype fabricated as a hollow

shell composed of several curved ribs that are connected at the base

by a circular ring and meet at the apex. One optical fiber with four

FBG sensors is embedded in the ribs. The structure is reinforced with

embedded carbon fibers. . . . . . . . . . . . . . . . . . . . . . . . . . 41

4.2 Miniaturized finger design (A) Prototype design with dimensions. (B)

Cross-sectional view cut by plane 1. (C) Cross-sectional view cut by

plane 2. (S1−S3: strain sensors, S4: temperature compensation sensor) 42

xiv

4.3 Experimental data of optical power loss for different bending radii for

a source in the 1550 nm wavelength window. . . . . . . . . . . . . . . 44

4.4 Modified SDM fabrication process for miniaturized finger prototype:

[Step 1] Carbon fiber pre-shaping (a) Prepare silicone rubber mold

to hold carbon fibers. (b) Place the carbon fibers and hold with

the mold. (c) Put small amount of cyanoacrylate glue on the carbon

fibers. (d) Remove the mold when the glue dries and trim unnecessary

part. [Step 2] Fingertip fabrication (a) Prepare patterned outer mold

(b) Place optical fibers with FBGs (c) Place pre-shaped carbon fibers.

(d) Place inner mold. (e) Cast liquid polyurethane. (f) Remove the

molds when the polyurethane cures. [Step 3] Joint fabrication. (a)

Prepare outer mold. (b) Cast liquid polyurethane. (c) Place cured

fingertip into the uncured polyurethane. (d) Remove the mold when

polyurethane cures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4.5 Silicone rubber molds and carbon fibers used for miniaturized finger

fabrication. (A) Carbon fibers on a silicone rubber mold for pre-

shaping. (B) Pre-shaped carbon fibers after removing the mold. (C)

Inner mold for fingertip casting. (D) Straight-line-patterned outer

mold for fingertip casting. . . . . . . . . . . . . . . . . . . . . . . . . 47

4.6 Force calibration setup. (A) x and y axes setup. (B) z-axis setup. . . 48

4.7 Calibration results. (A) x-axis force response (B) y-axis force re-

sponse. (C) z-axis force response. . . . . . . . . . . . . . . . . . . . . 49

4.8 FEM simulation results showing the strain difference for difference

force directions. (A) Straight rib design. (B) Pre-bent rib design.

The pre-bent rib design shows much higher z-axis force sensitivity

than the straight rib design. . . . . . . . . . . . . . . . . . . . . . . . 51

xv

4.9 Completed prototype of the miniaturized fingertip with pre-bent ribs.

The finger has 12 ribs, and carbon fibers were embedded in every other

ribs. Three FBG sensors were embedded in the pre-bent part of the

finger with 120◦ intervals. . . . . . . . . . . . . . . . . . . . . . . . . 52

4.10 Different types of machining for outer mold to show the feasibility.

(A) Not feasible: the tool cannot reach the pre-bent rib features. (B)

Not feasible: the tool cannot reach all the rib patterns. (C) Feasible:

the tool can reach all the features. . . . . . . . . . . . . . . . . . . . . 53

4.11 Outer mold assembly process. (A) Prepare six identical mold pieces.

(B) Place the six mold pieces in a radial shape. (C) Enclose the inner

mold with embedments. . . . . . . . . . . . . . . . . . . . . . . . . . 54

4.12 Pre-assembled molds. The inner mold holds optical firber with FBG

sensors and carbon fibers to be embedded. . . . . . . . . . . . . . . . 55

4.13 Calibration results. (A) x-axis force response. (y-axis response is

similar.) (B) z-axis force response. . . . . . . . . . . . . . . . . . . . 56

5.1 Needle deflection estimation model using two strain sensors in 2D

based on beam theory. . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.2 Four loading conditions considered for optimum sensor location de-

termination. (A) Two point loads. (B) One distributed load and

one point load. (C) One point load and one distributed load. (D)

Two distributed loads. (y1 and y2 are two sensor locations to be

determined, and L is the length of the needle.) . . . . . . . . . . . . . 64

5.3 Needle models with different loading conditions: (A) no load, (B) two

point loads in the same direction, and (C) two point loads in opposite

directions, where y1 and y2 are the first and second sensor locations,

and F1 and F2 are the magnitudes of two point loads at the mid-point

and the tip of the needle, respectively. . . . . . . . . . . . . . . . . . 65

xvi

5.4 Example of deflection estimation process based on beam theory. A

needle was fixed at the base (y = 0 mm), and two point loads of -0.6

N and 0.2 N were applied at the mid-point (y = 75 mm) and at the

tip (y = 150 mm) of the needle, respectively. . . . . . . . . . . . . . . 66

5.5 Tip position error plots for all possible sensor locations with two point

loads. (A) 3D contour plot. y1 and y2 represent the first and second

sensor locations, respectively. (B) 2D projection of the 3D contour

plot. Two white curves show possible sensor locations where the tip

position error is minimized. . . . . . . . . . . . . . . . . . . . . . . . 68

5.6 Tip position error plots for all possible sensor locations with one dis-

tributed and one point loads. (A) 3D contour plot. y1 and y2 rep-

resent the first and second sensor locations, respectively. (B) 2D

projection of the 3D contour plot. Two white curves show possible

sensor locations where the tip position error is minimized. . . . . . . 69

5.7 Tip position error plots for all possible sensor locations with one dis-

tributed load. (A) 3D contour plot. y1 and y2 represent the first and

second sensor locations, respectively. (B) 2D projection of the 3D

contour plot. The white curve shows possible sensor locations where

the tip position error is minimized. . . . . . . . . . . . . . . . . . . . 69

5.8 Tip position error plots for all possible sensor locations with two

distributed loads. (A) 3D contour plot. y1 and y2 represent the first

and second sensor locations, respectively. (B) 2D projection of the

3D contour plot. The white curve shows possible sensor locations

where the tip position error is minimized. . . . . . . . . . . . . . . . . 70

5.9 Superimposed plot of all three simulation results, Figures 5.5, 5.6,

5.8, showing optimal sensor sensor locations: y1=22 mm and y2=85

mm from the base of the needle. . . . . . . . . . . . . . . . . . . . . . 70

xvii

5.10 Prototype design with modified inner stylet incorporated with three

optical fibers. Three identical grooves with 120◦ intervals are made on

the inner stylet to embed optical fibers with FBGs along the needle

length. (A) Midpoint cross-section. (B) Magnified view of an actual

groove. (C) Tip of the stylet. . . . . . . . . . . . . . . . . . . . . . . 72

5.11 (A) Three-dimensional calibration setup with orthogonally placed

cameras and a light box. (B) Close up of the needle fixed to the

calibration apparatus. The tip was deflected in the x and z directions. 74

5.12 Wavelength shifts measured in experiment 1 (x-axis loading). . . . . . 75

5.13 Wavelength shifts measured in experiment 2 (z-axis loading). . . . . . 76

5.14 Wavelength measured in experiment 3 (temperature change). . . . . . 76

5.15 Assumptions for sensor location error calibration . . . . . . . . . . . . 78

5.16 Manufacturing error calibration result. (A) RMS value plot of tip

deflection error in two orthogonal planes, xy and yz. (B) Norm of

RMS values of tip deflection errors. . . . . . . . . . . . . . . . . . . . 79

5.17 Calibration result of all six FBG sensors for tip deflection. (A) Result

of Experiment 1 – deflection in yz plane. (B) Result of Experiment

2 – deflection in xy plane. . . . . . . . . . . . . . . . . . . . . . . . . 80

5.18 Screen capture of the display of the real-time monitoring system. . . . 83

5.19 (A) MRI scanned images with different deflections. The deflections

relative to MR images were found using a measurement tool in OsiriX

[79] software for viewing medical DICOM images. (B) Estimated

needle deflections using FBG sensors. . . . . . . . . . . . . . . . . . . 85

xviii

5.20 (A) 3T-MRI of the prototype needle in the prostate of dog in oblique

coronal (a, upper) and oblique sagittal (a, lower) views from refor-

matted 3D SPGR images. (B) Optically measured 3D estimation of

the needle shape. Axes scales are exaggerated to highlight bending.

Note the complete absence of any artifacts or interactions between

the MRI and the optical shape-sensing methods despite simultaneous

scanning and optical FBG interrogation. . . . . . . . . . . . . . . . . 86

xix

Chapter 1

Introduction

1.1 Motivation

Future robots are expected to free human operators from difficult and dangerous

tasks requiring dexterity in various environments. Prototypes of these robots already

exist for applications such as extra-vehicular repair of manned spacecraft and robotic

surgery, in which accurate manipulation is crucial. Ultimately, we envision robots

operating tools with levels of sensitivity, precision and responsiveness to unexpected

contacts that exceed the capabilities of humans, making use of numerous force and

contact sensors on their arms and fingers.

However, there are certain extreme environments where even robots are not

able to perform their assigned tasks without difficulties. One example of extreme

environments is space. Space is harsh and hostile in terms of radiation, extreme

ambient temperature ranges, and strong electro-magnetic interference from space

craft and large electromagnetic devices. Another example of a severe environment

is a magnetic resonance imaging (MRI). MRI-guided interventions have become

increasingly popular for minimally invasive treatments and diagnostic procedures

recently.

To tackle such applications it is desirable to develop a new method of sensing

1

CHAPTER 1. INTRODUCTION 2

that can be used without either affecting the performance of existing equipment or

being affected by interference from the environment, such as electromagnetic waves

and magnetic fields. Device size is another issue we need consider when we design

robots for special uses in certain extreme environments. For example, it is very

difficult to build a robot that fits inside the MRI bore due to the limited space once

we place a patient in it.

1.2 Thesis Outline

This thesis is organized into six chapters.

Chapter 2 provides background information regarding emerging needs in robot

sensing for performing special tasks in extreme environments, such as dexterous

manipulations for extra vehicular activities in space and robotic surgeries and inter-

ventional procedures in MRI facilities, where ferro-magnetic materials and electrical

circuits are not allowed because of the strong magnetic field. In addition, basic prin-

ciples and advantages of fiber optic strain sensors used for prototype development

are presented in this chapter. The chapter also provides background information on

shape deposition manufacturing, a rapid prototyping process used as an enabling

technology, discussing its basic process, benefits, and limitations.

Chapter 3 discusses the design and development of exoskeletal force sensing end-

effectors using embedded fiber optic strain sensors. The chapter starts with the main

design concepts and features of the prototype, and describes the fabrication process,

a modified and extended version of SDM. Next, the evaluation of the prototype is

discussed to show its performance as a force sensing structure in both static and

dynamic states. The chapter also discusses system integration and experimental

results of high-speed force and position control with the developed prototypes.

Chapter 4 describes the efforts of miniaturizing the force sensing robot fingers

discussed in Chapter 3. The chapter discusses the key design features of the minia-

turized force sensing finger and its fabrication process, another variation of SDM.


Force calibration results and suggestions for design improvement are also presented.

Chapter 5 discusses the design and development of an MRI-compatible bend-

shape and deflection sensing biopsy needle prototype. The modeling and physi-

cal design processes are presented in this chapter. Prototype fabrication using an

electro-discharge machining process is also described. The calibration procedures

and experimental results are provided for the evaluation of the prototype, and in-

vivo animal test results in an MRI system are presented.

Chapter 6 summarizes the results of this research and suggests future extensions

of this work.

1.3 Contributions

The main contribution of this thesis is the development of a new method of force

and position sensing for robots working in high magnetic fields, using embedded

fiber optic strain sensing, to provide real-time information to a user for high-speed

control and monitoring. Under this heading, specific contributions include:

• an exoskeletal force sensing robot structure with a complete immunity to

electro-magnetic interference. The design of the structure siginificantly sim-

plifies robot designs by embedding multiple sensors and reinforcing composite

materials. The new design enables building of robot parts that are light-weight

but relatively strong and robust, which provides higher flexibility and more

degrees-of-freedom in design. Also, the robot structures are independent of

electronics for signal conditioning and processing.

• an estimation scheme to localize contact forces in a hollow structure with

a limited number of sensors. The proposed scheme uses strain information

obtained from both global and local deformations of exoskeletal structures. It

enables estimation of the location of contact forces as well as measurement of

the magnitude of forces with a limited number (3-4) of strain sensors.


• a manufacturing method for fabricating a three-dimensional hollow structure

with embedded components and fibers. The proposed manufacturing method

is an extension of the conventional shape deposition manufacturing (SDM)

process, which enables fabrication of a fully three-dimensional structures with

complicated patterns, embedded sensors and fiber reinforcement. The process

allows building a structure to ”net shape” without direct machining on the

part.

• a method of instrumenting small diameter MRI-safe interventional tools. The

method, using a customized electro-discharging machining (EDM) process,

enables embedding of fiber optic strain sensors in interventional tools without

diminishing the tools’ structural integrity or MRI-compatibility.

Chapter 2

Background Information

2.1 Fiber Bragg Grating (FBG) Sensors

FBGs are sensor elements that are usually photo-written into germanium doped

optical fibers using periodic interference from intense ultra-violet laser beams [36].

The resulting diffraction gratings involving half-micron order periodic variation in

refractive index along the fiber core can be used for the measurement of strain (de-

formation or applied force [90]) and temperature, and when appropriately packaged

measurands such as acceleration [64, 92] and pressure [34, 94]. FBGs have reported

applications in civil engineering (in the monitoring of bridges, highways, and rail-

ways), aeronautics (in the monitoring of aerospace components) and biology (as

chemical and biological sensors).

The basic principle of an FBG-based sensor system lies in the monitoring of the

wavelength shift of the returned Bragg-signal, which is a function of the measur-

and (e.g., strain, temperature or force). Interrogation systems work by injecting

light from a spectrally broadband source into the fiber. When the light reaches the

grating, a narrow spectral component at the Bragg wavelength is reflected back.

In other words, this component is missing from the observed spectrum in trans-

mission, as shown schematically in Figure 2.1. A Bragg grating operates by acting

5

CHAPTER 2. BACKGROUND INFORMATION 6

Figure 2.1: Fiber Bragg grating structure with spectral response

as a wavelength selective filter that reflects a single wavelength, called the Bragg

wavelength, λB. The Bragg wavelength is related to the grating pitch, Λ, and the

effective refractive index of the core, neff , by λB = 2neffΛ.

When an FBG is subjected to strain or temperature changes, the reflected wave-

length changes. Measurement of these wavelength changes provides the basis for

strain and temperature sensing. Both the fiber refractive index (n) and the grating

pitch (Λ) vary with changes in strain (ε) and temperature (∆T ), such that the Bragg

wavelength shifts in response to longitudinal deformations in response to mechanical

or thermal effects. The change in the Bragg wavelength, ∆λB, is given by following

equation:

∆λBλB

= (1− pε)ε+ (αΛ − αN)∆T (2.1)

where pε is strain optic coefficient, αΛ is thermal expansion coefficient of fiber, and

αN is thermo optic coefficient.

The sensitivity of regular FBGs to axial strain is approximately 1.2 pm/µε at

1550 nm center wavelength [8, 50]. With the appropriate FBG interrogator, very


small strains, on the order of 0.1 µε, can be measured. In comparison to conventional

strain gages, this sensitivity allows FBG sensors to be used in sturdy structures that

experience modest stresses and strains under normal loading conditions. The strain

response of FBGs is linear with no indication of hysteresis at temperatures up to

370◦C [66] and, with appropriate processing, as high as 650◦C [62,69,103].

Among the advantages of FBG sensors are: immunity to electromagnetic inter-

ference (making them ideal of MR-applications), physical robustness (without com-

promising the bio-compatibility and sterilizability of the medical tools they modify),

and the ability to measure small strains. As a consequence of their ability to mea-

sure small strains, FBG sensors can be used directly without special features such

as holes or slots to increase strains in the vicinity of the gage. Other advantages

include the ability to place multiple FBG cells along a single fiber, reading each via

optical multiplexing, and the ability to use the same optical fibers for other sensing

modalities such as spectroscopy [95], optical coherence tomography, and fluoroscopy.

In conventional robotics applications, the chief drawback is that the optical inter-

rogator that reads the signals from the FBG cells is larger and more expensive than

the instrumentation used for foil or semiconductor strain gages. However the costs

of optical fiber interrogation systems are dropping steadily and in applications such

as MRI interventions, the capital costs are quickly amortized over many operations.

Although FBG sensors are a mature technology, innovations in photonics are

making it possible to read larger numbers of cells at higher sampling rates and with

smaller and less expensive equipment. Standard methods include Wavelength Divi-

sion Multiplexing (WDM), Time Division Multiplexing (TDM), Frequency Division

Multiplexing (FDM), Space Division Multiplexing (SDM) and CDM (Coherence Di-

vision Multiplexing) [32]. In the present work, we use a broadband light source and

optical wavelength division multiplexing so that all FBGs are read simultaneously

with each FBG sensor operating in a different slice of the available source spectrum.

The optical interrogator computes shifts in the wavelength of light returned by each

FBG, and reports these over a USB connection to our computer for calibration and


visualization.

2.2 Shape Deposition Manufacturing (SDM)

As an enabling technology for applying fiber optic sensors to a robotic system, we

employed the shape deposition manufacturing (SDM) rapid prototyping process [98].

SDM is a rapid prototyping process that enables fabrication of a part with desired

shapes by repeating deposition and machining steps, as shown in Figure 2.2. During

the repeated deposition steps, various components can be embedded inside the part.

Main benefits of SDM are:

• It is easy to make a part with heterogeneous materials without using fasteners

or adhesives.

• It is easy to build multiple parts with different shapes at the same time.

• It is easy to embed different parts, such as sensors or actuators, during fabri-

cation.

However, the conventional SDM process has some limitations. It becomes com-

plicated to build a fully three-dimensional part because the computer numerically

controlled (CNC) machine used for the machining process has direct access only

to the top of the part. Also, it is difficult to fabricate a hollow structure without

any opening because either the part material or the sacrificial material needs to be

removed eventually.

Therefore, a modified SDM process was developed to overcome these limitations.

The modification of the process extended the potential and application areas of

SDM.


Figure 2.2: SDM fabrication process with a continuous, alternating machining anddeposition cycles [4]

Chapter 3

Exoskeletal Force Sensing

End-Effectors with Embedded

Optical Fiber Bragg Grating

Sensors

3.1 Introduction

Compared to even the simplest of animals, today’s robots are impoverished in

terms of their sensing abilities. For example, a spider can contain as many as

325 mechanoreceptors on each leg [6, 28], in addition to hair sensors and chemi-

cal sensors [5, 84]. Mechanoreceptors such as the slit sensilla of spiders [6, 9] and

campaniform sensilla of insects [65, 87] are especially concentrated near the joints,

where they provide information about loads imposed on the limbs – whether due to

regular activity or unexpected events such as collisions. The slit sensilla is a small

mechanoreceptor in the exoskeleton of arthropods. It consists of a very narrow linear

opening in the exoskeleton that is connected to the creature’s nerve system, provid-

ing strain information caused by forces exerted on the structure. The campaniform

10

CHAPTER 3. EXOSKELETAL FORCE SENSING END-EFFECTORS 11

sensilla is another type of strain sensing mechanoreceptor also found in arthropods.

Both mechanoreceptors undergoe changes in opening geometry depending on strains

arising from by deformation, and the change is transmitted to the nerve system.

In contrast, robots generally have a modest number of sensors, often associated

with actuators or concentrated in special devices such as a force sensing wrist. Even

Robonaut, one of the most complex humanoid robots, has in its entire hand and

wrist module [2, 10, 59], only 42 sensors overall which include position and tem-

perature sensors as well as force and tactile sensors. The result of this typical

sensor improverishment in robots is that they often respond poorly to unexpected

and arbitrarily-located impacts. Although there have been some efforts to design

strain sensing structures inspired by one type of biological strain sensors of arthro-

pods (e.g., by campaniform sensilla [63,86,96]), the work focused on structures that

behave in the same way as campaniform sensilla, without trying to embed strain

sensors in a structure. The work in this chapter is part of a broader effort aimed

at creating light-weight, rugged appendages for robots that, like the exoskeleton of

an insect, feature embedded sensors so that the robot can be more aware of both

anticipated and unanticipated loads in real time.

Part of the reason for the sparseness of force and touch sensing in robotics is

that traditional metal and semiconductor strain gages are tedious to install and wire.

The wires are often a source of failure at joints and are receivers for electromagnetic

noise. The limitations are particularly severe for force and tactile sensors on the

fingers of a hand. Various groups have explored optical fibers for tactile sensing,

where the robustness of the optical fibers, the immunity to electromagnetic noise and

the ability to process information with a CCD or CMOS camera are advantageous

[47, 60, 78]. Optical fibers have also been used for measuring bending in the fingers

of a glove [41] or other flexible structures [16], where the light loss is a function of

the curvature. In addition, a single fiber can provide a high-bandwidth pathway for

taking tactile and force information down the robot arm [3].

We focus on a particular class of optical sensors, fiber Bragg grating (FBG)


sensors, which are finding increasing applications in structural health monitoring

[1, 51, 57] and other specialized applications in biomechanics [13, 17] and robotics

[70,73,74]. Examples include space or underwater robots [23,29,91], medical devices

(especially for use in MRI fields) [71,72,104], and force sensing on industrial robots

with large motors operating under pulse-width modulated control [25,101,105].

To our knowledge, the work in this chapter is the first application of FBG sen-

sors in hollow, bio-inspired multi-material robot limbs. The rest of the chapter is

organized as follows. Section 3.2 discusses design concepts for the force sensing fin-

ger prototype. Section 3.3 describes the fabrication process using a new variation

of a rapid prototyping process. Section 3.4 addresses the static and the dynamic

characterization of the sensorized finger structures, including the ability to local-

ize contact forces. Sections 3.5 and 3.6 describe the hand controller used with the

finger and the results of force control experiments. We conclude with a discussion

of future work, which includes a potential extension of the finger prototype with a

larger number of sensors for measurement of external forces and contact locations.

Future work also includes extending the capability of the optical interrogator and

using multi-core polymer fibers.

3.2 Design Concepts

Prototype fingers were designed as replacements for aluminum fingers on a two-

fingered dexterous hand used with an industrial robot for experiments on force

control and tactile sensing [33], as shown in Figure 3.1. Figure 3.2 shows a completed

finger prototype and design details are shown in Figure 3.3 including cross-sectional

views. Each of the two fingers can be divided into three parts: fingertip, shell,

and joint. The fingertip and shell are exoskeletal structures. Four FBG sensors are

embedded in the shell for strain measurement, and one FBG sensor is placed at the

center of the finger for temperature compensation. The remainder of this section

describes the design features of the prototype including the exoskeleton structure,


35

100

120

Diameter: 35

Thickness of shell: 2(unit: mm)

Adept Arm

Dexter Manipulator

FBG Embedded Force Sensing Finger

(A) (B)

Figure 3.1: (A) Prototype dimensions. (B) FBG embedded force sensing fingerprototypes integrated with two fingered hand and industrial robot.

solutions for reducing creep and the effects of temperature variations, and sensor

placement.

3.2.1 Exoskeleton Structure

In comparison to solid structures, exoskeletal structures have high specific stiffness

and strength. In addition, unlike a solid beam, they exhibit distinct local, as well

as global, responses to contact forces (Figure 3.4). This property facilitates the

estimation of contact locations. The exoskeletal structure may be compared with

the plastic fingertip described by Voyles et al. [97], which used electro-rheological

fluids and capacitive elements for extrinsic tactile sensing and required an additional

cantilever beam with strain gages for force-torque information.


Joint

Shell

Fingertip

Figure 3.2: Completed finger prototype. The prototype can be divided into threeparts: fingertip, shell, and joint.

To enhance the deformation in response to local contact forces, our exoskeleton is

designed as a grid. Although a grid structure with embedded FBG sensors has been

explored for structural health monitoring on a large scale [1], it has rarely been

considered in robotics. The ribs of the grid are thick enough to encapsulate the

optical fibers and undergo axial and bending strains as the grid deforms. Although

various polygonal patterns including triangles and squares are possible, hexagons

have the advantage of minimizing the ratio of perimeter to area [35,75] and thereby

reducing the weight of the part. Also, the hexagonal pattern avoids sharp interior

corners, which could reduce the fatigue life. The thickness of the shell and the width

of the pattern were determined so that each finger can withstand normal loads of

at least 12 N.


A

B B

A

S1

S2

S3

S4S5

Embedded

optical fibers

FBG sensor

Embedded

copper mesh

FBG sensor

Figure 3.3: Design details of the finger prototype with cross-sectional views (S1−S4:strain sensors, S5: temperature compensation sensor). See Table 3.2 for sensorparameters.

3.2.2 Creep Prevention and Thermal Shielding

Polymer structures experience greater creep than metal structures. Creep adversely

affects the linearity and repeatability of the sensor output. In addition, thermal

changes will affect the FBG signals. Drawing inspiration from a polymer hand

by Dollar et al. [21], a copper mesh (080X080C0055W36T, TWP Inc., Berkeley,

CA, USA) was embedded into the shell, to reduce creep and provide some thermal

shielding for the optical fibers. The high thermal conductivity of copper expedites

the distribution of heat applied from outside the shell and creates a more uniform


temperature within.

3.2.3 Strain Sensor Configuration

In general, larger numbers of sensors will provide more information and make the

system more accurate and reliable. However, since additional sensors increase the

cost and require more time and/or processing capacity, the optimal sensor config-

uration should be considered, as discussed by Bicchi [7]. In the present case, if we

assume that we have a single point of contact, there are five unknown values: the

longitude and latitude of a contact on the finger surface, and the three orthogonal

components of the contact force vector in the X, Y , and Z directions. For the initial

finger prototypes, we further simplify the problem by assuming the contact force is

normal to the finger surface (i.e., with negligible friction). This assumption reduces

the number of unknowns to three so that a minimum of three independent sensors

are needed. In the prototype, four strain sensors were embedded in the shell.

Before fabrication, finite element analysis was conducted to determine the sensor

locations. Figure 3.4 shows strain distributions when different types of forces are

applied to the shell and to the fingertip. Strain is concentrated at the top of the

shell where it is connected to the joint. The four sensors were embedded at 90◦

intervals into the first rib of the shell, closest to the joint, as shown in Figure 3.3.

3.2.4 Temperature Compensation

Since embedded FBG sensors are sensitive to temperature, it is necessary to isolate

thermal effects from mechanical strains. The sensitivity of regular FBGs to tem-

perature change is approximately 10 pm/◦C at 1550 nm center wavelength [36,46].

Various complicated temperature compensation methods have been proposed, such

as the use of dual-wavelength superimposed FBG sensors [99], saturated chirped

FBG sensors [100], and an FBG sensor rosette [61]. We chose a simpler method

that involved using an isolated, strain-free FBG sensor to measure thermal effects.


(A) (B)

Front View Right View Front View Right View

F

FF

F

Figure 3.4: Finite element models showing strain concentrations on the first ribclosest to the fixed joint. (A) Point load is applied to the fingertip. (B) Point loadis applied to the middle of the shell structure.

Subtracting the wavelength shift of this sensor from that of any other sensor corrects

for the thermal effects on the latter [77]. An important assumption in this method

is that all the sensors experience the same temperature. Our prototype has one

temperature compensation sensor in the hollow area inside the shell, as shown in

Figure 3.3. Although it is distanced from the strain sensors, the previously men-

tioned copper heat shield results in an approximately uniform temperature within

the shell. Since the temperature compensation sensor is encapsulated in a copper

tube attached at one end to the joint, it experiences no mechanical strain.


3.3 Prototype Fabrication

The finger prototype was fabricated using a variation of the SDM rapid-prototyping

process [98] to make a hollow three-dimensional part. The prototype was cast in a

three step process, shown in Figure 3.6, with no direct machining required.

3.3.1 Material Selection

The base material is polyurethane, chosen for its combination of fracture toughness,

ease of casting at room temperature and minimal shrinkage.

Different materials were considered for the prototype fabrication. The material

used in the early prototype (IE-72DC, Innovative Polymers, Saint Johns, Michigan,

USA) had reasonable stiffness and hardness. However, the mixed viscosity was too

high to remove air bubbles captured while curing. The material selected for the

final prototype (Task 3, Smooth-On, Easton, Pennsylvania, USA) had much lower

mixed viscosity with similar properties in hardness and stiffness, which facilitated

removing air bubbles with a vacuum degassing process. Table 3.1 compares the

properties different materials considered for prototyping. Figure 3.5 shows how

the new material improved the degassing process. The first prototype made of IE-

72DC, shown in Figure 3.5(A), showed a big air bubble with small bubbles in the

bonded area that might weaken the bonding. However, the second prototype made

of Task 3 had only small air bubbles, as shown in Figure 3.5 (B). In addition, Task

3 had a low mixed viscosity (150 cps), which helps it to completely fill the narrow

channels associated with ribs in the grid structure. Although Task 8 had the lowest

mixed viscosity (100 cps), the pot life (2.5 minutes) was too short for degassing and

enbedded part positioning.


Table 3.1: Material properties of polyurethane candidates.

IE-72DC IE-80D Task 3 Task 8 Task 9

Hardness 80 ± 5D 80 ± 5D 80D 80D 85D

Flexural Modulus (kpsi) 325 315 290 240 370

Mixed Viscosity (cps) 1600 400 150 100 300

Pot Life (min) 17.5 ± 2.5 15 ± 2 20 2.5 7

Demold Time 3 ± 1 hrs 5 ± 1 hrs 90 min 10–15 min 60 min

Manufacturer Innovative–Polymers Smooth–On

(B) Task 3

(Smooth-On)

(A) IE-72DC

(Innovative-Polymers)

Figure 3.5: Prototype comparison with different materials.


3.3.2 SDM Fabrication Process

The first step is to cast the shell (1.a-1.d in Figure 3.6). The outer mold is made of

hard wax to maintain the overall shape. The inner mold is hollow and made of sili-

cone rubber, which can be manually deformed and removed when the polyurethane

is cured. The optical fibers and copper mesh were embedded in this step. Although

it is often preferable to strip the 50µm polyimide coating on FBG regions before

optical fibers are embedded, we found that adequate bonding was obtained between

the polyurethane and the coated fibers, and the amount of creep was negligible

compared to overall deformation and creep in the urethane structure. Retaining the

coating also protected the fibers during the casting process.

The second step is fingertip casting (2.a-2.d), which uses separate molds and

occurs after the shell is cured. The polyurethane for the fingertip bonds to the

cured shell part.

In the final step, the joint is created (3.a-3.c). As with the fingertip, the joint

bonds to the cured shell. Since the joint is not hollow, an inner mold is not necessary.

Also, the joint is cast using hard polyurethane (Task 9) to reduce creep because it has

no copper mesh. In comparison, the shell and fingertip were cast using a somewhat

softer polyurethane (Task 3) to enhance impact resistance.

Figure 3.7 shows the molds and embedded copper mesh prepared for the modified

SDM process. After each step, the polyurethane is cured at room temperature for

2 to 3 days.

3.4 Static and Dynamic Characterization

The finger prototype was characterized with respect to static forces, modes of vi-

bration, hysteresis, and thermal effects.


Figure 3.6: Modified SDM fabrication process: [Step 1] Shell fabrication (a) Preparea silicone rubber inner mold and place optical fibers with FBG sensors. (b) Wrapthe inner mold with copper mesh. (c) Enclose the inner mold and copper mesh witha wax outer mold and pour liquid polyurethane. (d) Remove the inner and outermolds when the polyurethane cures. [Step 2] Fingertip fabrication (a) Prepare innerand outer molds and place copper mesh. (b) Cast liquid polyurethane. (c) Placethe cured shell into the uncured polyurethane. (d) Remove the molds when thepolyurethane cures. [Step 3] Joint fabrication (a) Prepare an outer mold and placea temperature compensation sensor structure. (b) Place the cured shell and fingertipinto the uncured polyurethane. (c) Remove the outer mold when polyurethane cures.


Table 3.2: Parameters of embedded FBG sensors

Sensor Wavelength Bandwidth Reflectivity

S1 1543.490 nm 0.380 nm 99.55 %

S2 1545.207 nm 0.360 nm 99.34 %

S3 1547.859 nm 0.370 nm 98.25 %

S4 1549.925 nm 0.310 nm 97.70 %

S5 1553.100 nm 0.400 nm 99.58 %

3.4.1 Static Force Sensing

Static forces were applied to two different locations on the shell and fingertip. Figure

3.8 shows the force locations and the responses of two sensors A and B, in the shell.

Applying forces to the shell yielded sensitivities of 24 pm/N and -4.4 pm/N for

sensors A and B, respectively. Sensor A, being on the same side of the shell as

the contact force, had a much higher strain. Applying a force to the fingertip

yielded sensitivities of 32 pm/N and -29 pm/N for sensors A and B, respectively.

In this case, the location of the force resulted in roughly equal strains at both

sensors. For a given location, the ratio of the sensor outputs is independent of

the magnitude of the applied force. The effect of location is discussed further in

Section 3.4.5. The optical interrogator can resolve wavelength changes of 0.5 pm or

less, corresponding to 0.02 N at the shell and 0.016 N at the fingertip. However,

considering the deviations from linear responses (RMS variations of 5.0 pm and 9.5

pm for the shell and the fingertip tests, respectively) the practical resolutions of

force measurement are estimated conservatively at 0.10 N at the shell and 0.15 N at

the fingertip1. The difference between the minimum detectable force changes and

the practical resolution for force sensing is due to a combination of effects including

1Over short time periods, higher accyracy is achieved, as seen in the force control results (Section3.6)


Figure 3.7: Wax and silicone rubber molds and copper mesh used in modified SDMfabrication process.

creep in the polymer structure, hysteresis and thermal drift over the 30 minute test

cycle. These effects are discussed further in the following sections.

3.4.2 Modes of Vibration

Prior to setting up a closed-loop control system, we investigated the dynamic re-

sponse of the fingers. Figure 3.9 shows the impulse response (expressed as a change


0 2 4 6 8 10 12-0.1

-0 .05

0

0.05

0.1

0.15

0.2

0.25

0.3

Sensor BW

avele

ng

th S

hif

t (n

m)

Applied Force (N)

Sensor A

Sensor B

*o

F

Sensor A

Fixed

Sensor

B

(A)

0 2 4 6 8 10 12-0.4

-0 .3

-0 .2

-0 .1

0

0.1

0.2

0.3

0.4

Applied Force (N)

F

Sensor

A

Sensor B

Wavele

ng

th S

hif

t (n

m)

(B)Sensor A

Sensor B

*o

Fixed

Figure 3.8: Static force response results. (A) Shell force response. (B) fingertipforce response.


in the wavelength of light reflected by an FBG cell) and its fast Fourier transform

(FFT). The impulse was effected by tapping on the finger with a light and stiff

object, a pencil. The FFT shows a dominant frequency around 167 Hz, which is a

result of the dominant vibration mode.

A finite element analysis (Figure 3.10) indicates that there are two dominant vi-

bration modes corresponding to the orthogonal X and Y bending axes, with nearly

equal predicted frequencies of just over 180 Hz. The difference between the com-

puted and measured frequency is due to the imperfect modeling of the local stiffness

of the polymer/mesh composite. The actual stiffness of the composite depends on

manufacturing tolerances including the location of the mesh fibers within the poly-

mer structure.

3.4.3 Hysteresis Analysis

Polymer structures in general are subject to a certain amount of creep and hysteresis,

which is one reason why they have traditionally been avoided for force sensing and

control applications. In the present case, these effects are mitigated by embedding a

copper mesh within the structure. However, there is still some creep and hysteresis

as shown in Figures 3.11 and 3.12. The plot in Figure 3.11 was produced by applying

a moderate load of approximately 1.8 N to the finger for several seconds and then

removing it suddenly. Figure 3.12 shows detailed views of loading and unloading

periods. The measured force was obtained by optically interrogating the calibrated

FBG sensors.

When a steady load is applied for several seconds there is a small amount of

creep, part of which also arises from imperfect thermal compensation. The effect

is relatively small over periods of a few seconds, corresponding to typical grasping

durations in a pick-and-place or manipulation task. A more significant effect occurs

when the load is released. As the plot indicates in Figure 3.12 (B), the force quickly

drops to a value of approximately 0.1 N and then more slowly approaches zero.


0 0.05 0.1 0.15 0.2 0.25 0.3 0.35-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

Time (Sec)

Wa

ve

len

gth

Ch

an

ge

(n

m)

0 200 400 600 800 10000

0.005

0.01

0.015

0.02

0.025

0.03

Frequency (Hz)

|Y(f

)|

(A)

(B)

Frequency [Hz]

Time [Sec]

Wavele

ngth

shift [n

m]

| Y

(f)

|

Figure 3.9: (A) Impulse response of the finger prototype. (B) Fast Fourier transformof impulse response.


941.5 Hz938.7 Hz479.1 Hz185.1 Hz181.2 HzFrequency

54321

Mode

Figure 3.10: Modes of vibration of the finger prototype using finite element analysis.Modes 1 and 2 are the dominant modes, representing bending about x and y axes,respectively.

To overcome this effect in manipulation tasks, a simple strategy was employed.

Whenever the force suddenly dropped to a small value (less than 0.17 N), we assumed

that contact had been broken. At this point, we reset the zero-offset after a brief

time delay. As described in the following section, loss of contact is also a signal to

switch the robot from force control to position control.

3.4.4 Temperature Compensation

Figure 3.13 shows a typical thermal test result. Over a three minute period, the fin-

gertip was loaded and unloaded while the temperature was decreased from 28.3◦C to

25.7◦C. The ideal (temperature invariant) sensor output is indicated by the dashed

line. The results show that the temperature compensation sensor reduces thermal

effects. However, a more accurate compensation design is desired in the next pro-

totype.

3.4.5 Contact Force Localization

It is useful to know the locations of contact forces when a robot is manipulating an

object. It is also useful to distinguish, for example, between a desired contact on


0 5 10 15 20 25 30 35 40-0.5

0

0.5

1

1.5

2

2.5

Time [Sec]

Forc

e [N

]

Creep

Figure 3.11: The effect of applying a steady load for several seconds and suddenlyremoving it from the polymer fingertip.

6 8 10 121.65

1.7

1.75

1.8

1.85

25 30 35 40

-0.05

0

0.05

0.1

0.15

0.2

Time [Sec]

Forc

e [N

]

Time [Sec]

Forc

e [N

]

(A) (B)

Figure 3.12: Detailed views of creep under steady loading (A) and of the hysteresisassociated with sudden unloading (B).


Temperature compensated force response

Temperature uncompensated force response

Ideal force response

Figure 3.13: Test result showing partial temperature compensation provided by thecentral sensor.

the fingertip and an unexpected contact elsewhere on the finger. Since the finger

prototype has a cylindrical external shape, the location of a contact force can be

expressed in terms of latitude and longitude. The following discussion assumes a

single contact.

Longitudinal Location

Longitudinal localization requires some understanding of the structural deformation

of the shell. Figure 3.14 shows simplified two-dimensional diagrams of the prototype.

When a force is exerted at a certain location, as shown in (A), the structure will

deform and sensors A and B will measure strains εA and εB, respectively as indicated.

This situation can be decomposed into two separate effects, as shown in (B) and


Fld_(C)

(A)

εεεε3

εεεε2

F(B)

εεεε1

F

l

εεεεB (Sensor B)

d

εεεεA (Sensor A)

++++––––

Figure 3.14: 2D simplified shell structure and deformations of finger prototype.

(C). By superposition, εA = ε1 + ε2 and εB = ε3. Therefore, if the ratio of εA to εB

is known, we can estimate d, the longitudinal force location. Figure 3.15 shows the

plot of experimental ratios of εA to εB as a function of d.

There is some ambiguity in the localization, since two values of d result in the

same ratio. However, if we let d0 be the distance at which εA/εB is minimized, and

we restrict ourselves to the region d > d0, we can resolve this ambiguity. Further, if

we modify the manufacturing process to place the sensors closer to the other surface

of the shell, d0 approaches 0 and we can localize an applied force closer to the joint.

Latitudinal Location

Latitudinal location can be approximated using centroid and peak detection as dis-

cussed by Son et al. [88]. Figure 3.16 (A) shows a cross-sectional view of the finger

with four strain sensors and an applied contact force indicated. Figure 3.16 (B)

shows its corresponding sensor signal outputs. The two sensors closest to the force

location will experience positive strains (positive sensor output), and the other two


0 10 20 30 40 50 60 70

-3.2

-3

-2.8

-2.6

-2.4

-2.2

-2

-1.8

-1.6

-1.4S

train

Ratio (εε εε

A/ εε εε

B)

Distance from Joint (d) [mm]

Figure 3.15: Strain ratio plot of sensor A to B (εA/εB) with error estimates forseveral locations of force application along the length of the finger.

sensors will experience negative strains (negative sensor output), regardless of the

longitudinal location of the force, if d > d0. However, since all the sensor signals

must be non-negative to use the centroid method, all signal values must have the

minimum signal value subtracted from them. With this provision, we can find the

angular orientation, θ, of the contact force:

θ =

∑φiS

′i∑

S ′i− α (3.1)

for i = 1, 2, 3, 4, where S ′i = Si − min{S1, S2, S3, S4}, φ1 = α and φk = φk−1 + π2,

for k = 2, 3, 4 (if φk ≥ 2π, φk = φk − 2π), Si is the output signal from sensor i, and

α is the clockwise angle between sensor 1 and the sensor with the minimum output


θθθθ

F Sensor 1

Sensor 2

Sensor 3

Sensor 4

φφφφi

Si'

Sen

so

r O

utp

ut

F

ππππ 3ππππ

2 2 0 ππππ

S1

S4

S3

S2

(A) (B)

αααα

Figure 3.16: (A) Top view of the prototype showing embedded sensors and forceapplication. (B) Plot of sensor signal outputs.

signal value.

This centroid and peak detection method produced errors of less than 2◦, cor-

responding to less than 0.5 mm on the perimeter in both FEM simulation and

experiments. However, the experimental data yielded an offset of approximately 5◦

while the simulation data yielded an offset of approximately 1.5◦. The difference is

likely due to manufacturing tolerances in the placement of the sensors.

3.5 Force Controller

Figure 3.17 shows the architecture of the hardware system. The two-fingered robot

hand, Dexter, is a low-friction, low-inertia device designed for accurate force control.

The hand is controlled by a process running under a real-time operating system

(QNX) at 1000 Hz, which reads the joint encoders, computes kinematic and dynamic

terms and produces voltages for linear current amplifiers that drive the motors [33].

The hand controller also acquires force information, via shared memory, from a

process that obtains analog force information at 5 kHz from the optical interrogator

(I*Sense, IFOS Inc., Santa Clara, CA, USA) that monitors FBG sensors.


Adept Control

Ethernet @ 62.5 Hz

Dexter Control

Servo Card @ 1000 Hz

Digital Force Input

Servo Card @ 1000 Hz

Analog Force Input

Interrogator @ 5000 Hz

Servo Card

FBG Interrogator

QNX System

Adept Robot

Dexter

FBG Fingers

FBG Optical Signal

Figure 3.17: Hardware system architecture.

The FBG interrogator is based on high-speed parallel processing using Wave-

length Division Multiplexing (WDM). Multiple FBG sensors are addressed by spec-

tral slicing, with the available source spectrum divided up so that each sensor is

addressed by a different part of the spectrum. The interrogator built for this work

uses sixteen channels of a parallel optical processing chip. Each channel is sepa-

rated by 100 GHz (approximately 0.8 nm wavelength spacing around an operating

wavelength of 1550 nm)1 so that the total required source bandwidth is 12.8 nm.

Dexter is mounted to a commercial AdeptOne-MV 5-axis industrial robot. Com-

munication with the Adept robot is performed using the ALTER software package,

which allows new positions to be sent to the Adept robot over an Ethernet connec-

tion every 16 ms (62.5 Hz). Due to this limitation, all force control is done within

1Operation is in the 1550-nm wavelength window (and, more specifically, within the C-band)to exploit the availability and low cost of components for telecom applications.


Dexter, and the Adept robot is used only for large motions and to keep Dexter

approximately centered in the middle of the workspace.

When the fingers are not in contact with an object, the fingers are operated

under computed-torque position control, with real-time compensation for gravity

torques and inertial terms. When in contact, the fingers are switched over to a

nonlinear force control as described in the next section.

3.6 Contact force control

Most implementations of contact force control can be divided into two categories:

impedance control and direct force control [102]. The impedance control [37–39,49]

aims at controlling position and force by establishing desired contact dynamics.

Force control [76] commands the system to track a force setpoint directly. For this

work, we adopted a nonlinear controller presented by our collaborator at NASA,

the late H. Seraji [81–83]. When the system detects contact with the fingertip,

it switches to force control as depicted in Figure 3.18. The system actually per-

forms hybrid force/position control [58, 76] at this stage, as the position and force

controllers are combined to control forces. The proportional-integral (PI) force con-

troller is constructed as

K(s) = kp +kis

(3.2)

based on the first-order admittance

Y (s) = kps+ ki (3.3)

where kp and ki are the proportional and integral force feedback gains, respectively.

To make the force controller simple, we fix the proportional gain, kp, to a constant

and make the integral gain, ki, a nonlinear function of the force error. The nonlinear


−

rX

+

Force

Controller

Position

ControllerDEXTER

Force Sensor

rF

fX +

+−

Encoder

F

F

X

cX

X

Figure 3.18: Position based force control system. F and Fr are contact force anduser-specified force setpoint. X, Xc, Xf , and Xr are respectively actual position,commanded position, position perturbation computed by the force controller, andreference position of the end-effector.

integral gain is determined by the sigmoidal function

ki = k0 +k1

1 + exp[−sgn(∆)k2e](3.4)

where e is the force error (Fr−F ), ∆ = Fr−Fs, Fs is the steady value of the contact

force before applying new Fr and k0, k1, and k2 are user-specified positive constants

that determine the minimum value, the range of variation, and the rate of variation

of ki, respectively. The value of sgn(∆) is +1 when Fr > Fs, and -1 when Fr < Fs.

We can achieve fast responses and small oscillations in control with this nonlinear

gain since the nonlinearity provides high gains with large errors and low gains with

small errors. To minimize oscillations due to large proportional gains when the

switch occurs between position and force control, all gains except the integral force

feedback gain are ramped from zero to the defined values over a transition time of

0.1 seconds.


3.6.1 Results of Experiments

In this section, we present the results of two experiments that assess the accuracy

of control achieved with the finger prototype. The first experiment shows how accu-

rately the manipulator maintains a desired force during contact by comparing the

force data from the prototype with that from a commercial 6-axis force-torque sen-

sor (ATI-Nano25 from ATI Industrial Automation). The second experiment shows

force control during manipulation tasks, including linear and rotational motions of

the hand, while grasping an object.

Experiment 1 (Force Setpoint Tracking)

The Adept arm moves in one direction until the fingertip touches the commercial

load cell. As soon as the finger detects contact, the Adept arm stops and the Dexter

hand switches to force control. After a period of time, the Adept arm moves away

from the object and the hand switches back to position control. Figure 3.19 shows

the horizontal motion of the Adept arm in parallel with the joint rotation of the

distal joint of the Dexter hand and the force data from both the finger and the load

cell. The result shows the force data from the finger and the load cell almost match

exactly over the duration of the experiment. In addition, there is a small amount of

slippage reflected in the mirror-image dynamic force signals reported by the finger

and load cell, respectively, as the finger breaks the contact.

We note that to complete the experiment it was necessary to carefully shield and

ground all wires emanating from the commercial load cell due to the large magnetic

fields produced by the industrial robot.

Experiment 2 (Force Control during Manipulation)

This experiment concerns the ability of the hand to maintain a desired grasp force

while subject to motions in a manipulation task. The robot was commanded to lift

the grasped object, a metal block weighing 100 g, move it horizontally a distance


FBG Finger

Load Cell

(A)

(B)

(C)

t3t2t1

Figure 3.19: Experimental results of force setpoint tracking. (A) Adept robot mo-tion. (B) Joint angle change of Dexter manipulator. (C) Force data from load celland FBG embedded robot finger prototype. Robot starts force control as soon as itmakes a contact with the object at t1. Robot starts to retreat at t2. Robot breakscontact at t3.


of approximately 30 cm, rotate it about the Z and Y axes, return the block to the

original location, and replace it. In every case, the controller returned to the desired

force within 0.01 seconds. The results of this experiment can be seen in Figure 3.20.

The magnitude of the combined (x, y, and z) acceleration of the manipulator is

plotted in parallel with the measured grasp force. Disturbances associated with the

accelerations and decelerations along the path can be observed in the force data.

The root-mean-square of force errors during the force control is < 0.03 N.

Since the current finger prototype is capable of control one-axis forces, more

complicated force control experiments, in two or three axes, will be carried out in

the future.

3.7 Conclusions and Future Work

This chapter described composite robot end-effectors that incorporate optical fibers

for accurate force sensing and control and for estimating contact locations. The

overall design is inspired by biological mechanoreceptors, such as slit sensillae or

campaniform sensillae, in arthropod exoskeletons that allow them to sense contacts

and loads on their limbs by detecting strains caused by structural deformation.

A new fabrication process is proposed to create multi-material reinforced robotic

structures with embedded fibers. The results of experiments are presented for char-

acterizing the sensors and controlling contact forces in a closed loop system involving

a large industrial robot and a two fingered dexterous hand. The proposed exsoskele-

ton finger structure was able to detect less than 0.02 N of contact force changes

and to measure less than 0.15 N of contact forces practically. A brief description of

the optical interrogation method, used for measuring multiple sensors along a single

fiber at kHz rates needed for closed-loop force control, was also provided.


f

0

a b c d e

Force setpoint level (A)

(B)

Figure 3.20: Experimental results of force control during manipulation tasks (A)Grasp force measured by a finger with FBG sensors (B) Acceleration plotted alongwith magnitude of combined (x, y, and z) acceleration of the robot. Periods a, b, e,and f are for translation motions. Periods c and d are for rotation motions. Everytask motion is followed by a waiting period before starting next motions.

Chapter 4

Miniaturized Force Sensing

Fingers

4.1 Introduction

Following the successful creation of large-scale (120 mm long) robot fingers, the next

step was to produce human-scale fingertips for robots designed for human interaction

in space. The same technology, having no metal components or electronics, could

also be applied to robots for MRI procedures.

Figure 4.1 shows a prototype of a small fingertip with an embedded optical

fiber containing FBG strain sensors. For this application, an 80 µm diameter bend-

insensitive optical fiber was selected. These fibers are designed to tolerate compar-

atively tight bending radii (approximately 5 mm). In addition to the optical fibers,

carbon fibers were embedded for structural reinforcement and creep reduction.

4.2 Design Concepts

Figure 4.2 shows the design details of the prototype. Three FBG sensors for force

measurement and one for temperature compensation were embedded with carbon

40

CHAPTER 4. MINIATURIZED FORCE SENSING FINGERS 41

10 mm 15 mmEmbedded

optical fibers Joint Fingertip

Embedded

carbon fibers

Figure 4.1: Miniaturized polyurethane finger prototype fabricated as a hollow shellcomposed of several curved ribs that are connected at the base by a circular ringand meet at the apex. One optical fiber with four FBG sensors is embedded in theribs. The structure is reinforced with embedded carbon fibers.


Figure 4.2: Miniaturized finger design (A) Prototype design with dimensions. (B)Cross-sectional view cut by plane 1. (C) Cross-sectional view cut by plane 2. (S1−S3: strain sensors, S4: temperature compensation sensor)

fibers in a straight-line-patterned exoskeletal structure.

4.2.1 Bend-insenstive Optical Fibers

The optical fiber embedded in the miniaturized finger prototype is an bend-insensitive

fiber (diameter: 80 µm) from OFS, while the optical fiber used for the larger fingers

was a standard fiber (diameter: 125 µm) [44]. One problem of standard fibers is


that optical loss exponentially increases as the bending radius becomes less than

12.5 mm [44] for the wavelength of interest around 1550 nm.

Since the radius of the small finger is 5 mm, the round-shaped fingertip region

caused too much optical power loss with standard fibers, making it difficult to read

multiple FBG sensors from one end of the fiber simultaneously. The bend-insensitive

fiber was able to transmit and receive optical signals to and from all the embedded

sensors without any significant power loss. This is due to SMF-28 having been

designed to be single moded at the 1330 nm and 1550 nm. (Bending loss increases

for smaller guidance parameter as well as smaller bend radii.) To be single-moded,

a step index fiber requires the guidance parameters V = (2πρ/λ)(n2co− n2

cl)1/2 to be

less than the second mode cutoff values of Ve ≈ 2.4. For standard Corning SMF-28,

seeing that the corresponding second mode curoff wavelength is less than 1300 nm

(around 1250 nm), at 1550 nm, the guidance parameter is V ≈ (1250/1550)·2.4 =

1.94. As OFS fiber was designed to operate close to second mode cutoff 1550 nm,

its guidance parmeter at 1550 nm is larger resulting in less bending loss.

Figure 4.3 compares the optical power losses with different bending radii between

standard fibers and bend-insensitive fibers. Optical losses were measured after mak-

ing 5 and 10 turns on a solid rod with different radius for each test. The plot shows

the insertion loss rapidly increases as the bending radius becomes less than 10 mm

for standard fibers and less than 4 mm for bend-insensitive fibers. Considering the

radius of the miniaturized fingertip, 5 mm, 80 µm bend-insensitive fibers were the

right selection for embedment.

4.2.2 Structure Design

The miniaturized fingers have similar exoskeletal structures to those of the larger

fingers. The difference is straight-line pattens are used on the structure, instead of

hexagonal patterns. Since the structure is at a smaller scale, as shown in Figure

4.2, it was difficult to embed optical fibers in hexagonal patterns. Considering the


0

5

10

15

20

25

30

35

40

0 5 10 15 20 25

Bend-radius (mm)

Ins

ert

ion

Lo

ss

(d

B)

Bend radius [mm]

Optical pow

er

loss [dB

]

Standard fiber (5 turns)

Standard fiber (10 turns)

Bend-insensitive fiber (5 turns)

Bend-insensitive fiber (10 turns)

Figure 4.3: Experimental data of optical power loss for different bending radii for asource in the 1550 nm wavelength window.

length of the embedded FBG sensors, approximately 5 mm, which is almost half of

the entire length of the fingertip, the structure was designed to be a straight-line-

patterned exsoskeleton.

4.2.3 Sensor Configuration

Since the finger prototypes are measuring three-axial forces (x, y, and z), at least

three FBG sensors, for strain measurement, need to be embedded. Similar to larger

fingers, the three FBG sensors were embedded in the ribs close to the joint with 120◦

intervals, as shown in Figure 4.2. In addition to the three FBG sensors, one FBG

sensor, for temperature compensation, was placed at the center of the fingertip. The


independence of the temperature compensation sensor from the fingertip structure

ensures no mechanical strains applied to the sensor.

4.2.4 Creep Prevention

The creep effect of a polymer-based structure needed to be taken care of in the

miniaturized finger prototype too. Instead of copper mesh, used for larger fingers,

carbon fiber (530, Fibre Glast Developments Corp., Brookville, OH, USA) was em-

bedded for creep reduction because copper mesh was too thick to embed in such

a narrow and small structure. Since carbon fibers, used in this prototype, do not

have as high a thermal conductivity as that of copper mesh, they do not provide

the same thermal shielding effect that the larger fingers have. However, the smaller

dimensions of the miniaturized fingertips allow the FBG sensors to be located much

closer each other, which helps them to be in a uniform thermal area.

4.3 SDM Fabrication Process

The main difference between the miniaturized finger prototypes and the larger fin-

gers is carbon fiber embedment for creep reduction instead of copper mesh. Since

the carbon fibers are very soft and unshaped before being embedded, there is an ad-

ditional process to pre-shape carbon fibers before actual finger fabrication, as shown

in Figure 4.4. The base material is the same polyurethane as with the larger fingers.

The first step is to pre-shape carbon fibers (1.a-1.d in Figure 4.4). The carbon

fibers need to be fixed to a custom-made silicone rubber mold, and cyanoacrylate

glue is applied. When the glue cures, the carbon fibers are hardened and remain in

the current shape even though the mold is removed later.

The second step is to cast the fingertip (2.a-2.f in Figure 3.6). Both the outer and

inner molds are made of silicone rubber. The outer mold has straight-line patterns

where the optical fibers are embedded. The optical fibers with FBG sensors and


Figure 4.4: Modified SDM fabrication process for miniaturized finger prototype:[Step 1] Carbon fiber pre-shaping (a) Prepare silicone rubber mold to hold carbonfibers. (b) Place the carbon fibers and hold with the mold. (c) Put small amount ofcyanoacrylate glue on the carbon fibers. (d) Remove the mold when the glue driesand trim unnecessary part. [Step 2] Fingertip fabrication (a) Prepare patternedouter mold (b) Place optical fibers with FBGs (c) Place pre-shaped carbon fibers.(d) Place inner mold. (e) Cast liquid polyurethane. (f) Remove the molds when thepolyurethane cures. [Step 3] Joint fabrication. (a) Prepare outer mold. (b) Castliquid polyurethane. (c) Place cured fingertip into the uncured polyurethane. (d)Remove the mold when polyurethane cures.


(A) (B) (C) (D)

Figure 4.5: Silicone rubber molds and carbon fibers used for miniaturized fingerfabrication. (A) Carbon fibers on a silicone rubber mold for pre-shaping. (B) Pre-shaped carbon fibers after removing the mold. (C) Inner mold for fingertip casting.(D) Straight-line-patterned outer mold for fingertip casting.

pre-shaped carbon fibers are embedded in this step. 80µm diameter bend-insensitive

optical fibers are used in this step instead of reqular 125 µm diameter regular fibers

due to the small radius of the finger tip.

In the final step, the joint is created (3.a-3.d). The joint bonds to the cured

fingertip while curing. Figure 4.5 shows the molds and embedded carbon fibers

prepared for the miniaturized finger fabrication. After each step, the polyurethane

is cured at room temperature for 2 to 3 days.

4.4 Force Calibration

Force calibration tests were conducted along all three axes (x, y, and z). Vertical

forces were applied to the finger with different configurations, depending on calibra-

tion axes, using a height gauge, and the applied forces were measured with a digital

scale, as shown in Figure 4.6. The resolution of the digital scale is 0.01 g.

Figure 4.7 shows the results of force calibration tests. Applying force up to ap-

proximately 5 N to the fingertip yielded sensitivities of 71 pm/N, 54 pm/N, and -7.2


Figure 4.6: Force calibration setup. (A) x and y axes setup. (B) z-axis setup.

pm/N in x, y, and z directions, respectively. Considering the wavelength resolu-

tion of the optical interrogator, better than 0.5 pm, the minimum detectable force

changes are less than 0.01 N in the x and y directions and 0.07 N in z direction

assuming no temperature changes. The practical resolutions of force measurement

are 0.05 N in x and y and 0.16 N in z considering deviations from linearity.

4.5 Improvement of z-axis Force Sensitivity

The reason for the much coarser resolution of z-axis force measurement is that

the axial strain caused by pure compression is much smaller than that caused by

bending. The structure can be improved to give higher sensor signal in z-axis by

modifying the physical design.


0 1 2 3 4 5-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

Force (N)

sensor 1

sensor 2

sensor 3

0 1 2 3 4 5-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Force (N)

sensor 1

sensor 2

sensor 3

0 1 2 3 4 5-0.04

-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

Force (N)

sensor 1

sensor 2

sensor 3

x

z

y +Fx

x

z

y

+Fy

x

z

y -Fz

(A)

(B)

(C)

Force [N]

Force [N]

Force [N]

Wa

ve

len

gth

shift

[nm

]W

ave

len

gth

shift

[nm

]W

ave

len

gth

shift

[nm

]

Figure 4.7: Calibration results. (A) x-axis force response (B) y-axis force response.(C) z-axis force response.


4.5.1 Design Modification

Figure 4.8 (A) shows the FEM simulation result of the previous straight-rib finger

design with different force directions. The simulation result shows that the axial

strain caused by forces applied transverse to the finger is approximately 27 times

higher than that caused by forces applied axially. However, we can improve the

strain signal from the axial loads by having pre-bent ribs near the base of the finger,

where FBG sensors are embedded, as shown in Figure 4.8 (B). The new design

makes the ribs bend, instead of being compressed axially, even with axial loads as

well as with transverse loads. The FEM simulation result shows that the strain

from the transverse loads is just 2.5 times higher than that from axial loads with

the new pre-bent rib design. The actual prototype with pre-bent rib design is shown

in Figure 4.9.

4.5.2 SDM Fabrication Process

The prototype fabrication process for the pre-bent rib design is more complicated

than the modified SDM process discussed so far. We needed to further modify the

SDM process. The biggest challenge in this design was machining the outer mold

that had pre-bent rib features. The pre-bent rib feature prevented machining from

the top of the mold, and the rib patterns made it difficult to machine from the side of

the mold, as shown in Figures 4.10 (A) and (B), respectively. Therefore, we decided

to subdivide the mold into six pieces, each having the shape with top view shown in

Figure 4.10 (C). This subdivision enabled the tool to access all the features from the

top only. Figure 4.11 shows the actual outer molds and the assembly process, and

Figure 4.12 shows the pre-assembled mold showing the inner mold and the carbon

fiber embedment before pouring liquid polyurethane.


(A)

(B)

Figure 4.8: FEM simulation results showing the strain difference for difference forcedirections. (A) Straight rib design. (B) Pre-bent rib design. The pre-bent rib designshows much higher z-axis force sensitivity than the straight rib design.


Optical Fibers

Pre-bent ribsEmbedded

carbon fiber

30 mm

10 mm

Figure 4.9: Completed prototype of the miniaturized fingertip with pre-bent ribs.The finger has 12 ribs, and carbon fibers were embedded in every other ribs. ThreeFBG sensors were embedded in the pre-bent part of the finger with 120◦ intervals.

4.5.3 Force Calibration

Figure 4.13 shows the preliminary force calibration results for transverse (x-axis) and

axial (z-axis) loads using the same loading configurations as discussed in Section 4.4

(see Figure 4.6). Applying force up to approximately 11 N to the fingertip yielded

sensitivities of -24 pm/N and -7.1 pm/N in x and z directions, respectively. With

the pre-bent rib design, the sensitivity to x-axis loading was just 3.2 times higher

than that to z-axis loading, while the sensitivity to x-axis loading was almost 10

times of that to z-axis loading with a straight rib design. The reason that the

absolute values of measured z-axis sensitivity look similar in both the straight rib

and pre-bent rib designs, is that the pre-bent rib prototype has much stronger carbon

fiber reinforcement for higher force application. If the prototype were made with


Figure 4.10: Different types of machining for outer mold to show the feasibility.(A) Not feasible: the tool cannot reach the pre-bent rib features. (B) Not feasible:the tool cannot reach all the rib patterns. (C) Feasible: the tool can reach all thefeatures.

same dimensions and same amount of reinforcement, the pre-bent rib design would

provide much higher sensitivity to z-axis load, as shown in FEM analysis in Figure

4.8.


A miniaturized (human size) force sensing robot finger was designed and developed

by embedding four FBG sensors inscribed in 80 µm bend-insensitive optical fiber.

The miniaturized finger prototype showed the capability of three-axis force sens-

ing. The resolutions of force measurement were 0.05 N in x and y and 0.16 N in

z. Carbon fibers were embedded in addition to the optical fiber for reinforcement

to deal with the creep effect of a polymer structure. A modified design achieved

greater sensitivity to axial loads. A modified SDM process successfully enabled the

fabrication of human scale exoskeletal robot fingertips with embedded fiber optic


(A)

(B)(C)

Figure 4.11: Outer mold assembly process. (A) Prepare six identical mold pieces.(B) Place the six mold pieces in a radial shape. (C) Enclose the inner mold withembedments.


Figure 4.12: Pre-assembled molds. The inner mold holds optical firber with FBGsensors and carbon fibers to be embedded.

sensors. Future versions of this prototype will incorporate additional sensors for

more accurate thermal compensation.


0 2 4 6 8 10 12-0.09

-0.08

-0.07

-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

Sensor 1

Sensor 2

Sensor 3

0 2 4 6 8 10 12-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Sensor 1

Sensor2

Sensor 3

+Fx

-Fz

Force [N]

Force [N]

Wavele

ngth

shift [n

m]

Wavele

ngth

shift [n

m]

(A)

(B)

Figure 4.13: Calibration results. (A) x-axis force response. (y-axis response issimilar.) (B) z-axis force response.

Chapter 5

Real-Time Estimation of

Three-Dimensional Needle Shape

and Deflection for MRI-Guided

Interventions

5.1 Introduction

Magnetic resonance imaging (MRI) guided interventions have become increasingly

popular for minimally invasive treatments and diagnostic procedures. However, cur-

rent hardware and software capabilities of MRI systems result in iterative processes

of moving the patient in and out of the scanner for imaging and intervention [19,53].

Furthermore, visualization of the entire minimally invasive tool and its environmen-

tal interactions is limited, particularly when there is uncertainty about the endpoint

locations of slender tools.

During MRI-guided breast and prostate biopsies, radiologists note mild to sig-

nificant needle bending (2−20 mm in tip deflections). These deflections, which are

not immediately recognizable without visual feedback, may necessitate reinserting

57

CHAPTER 5. MRI-COMPATIBLE SHAPE SENSING NEEDLE 58

the needle to reach a desired target.

Methods in active tracking of devices in MRI environments [18, 22, 43, 85] are

increasingly fast and accurate, yet these techniques, as reviewed in [19], have limi-

tations in regards to line-of-sight, heating, sensitive tuning, complex calibration and

expense. Passive tracking methods [20] rely on the observing the device and pa-

tient’s anatomy together as well as the use of bulky stereotactic frames or external

fiducials [27,48]. The use of RF coils [56] and rapid MR tracking [52] techniques are

also limited by the need for continual use of the scanner in order to image and visu-

alize the devices. Another issue with current tracking methods is that they require

sensors which are, in general, too large for incorporation in a needle [11].

We developed an instrumented needle which incorporates optical fibers with fiber

Bragg gratings (FBGs) for measuring strain. In other medical applications, FBG

sensors have been embedded into catheters [24] and endoscopes [104] for shape

sensing. Also, FBG sensors have been used to measure microforces of a retinal

surgery tool [45, 89]. To our knowledge, this is the first application of FBG sensing

in a small gauge MRI-compatible biopsy needle. We have presented the feasibility

of using FBG sensors in MR-interventions [71, 72] and, in this chapter, will explain

the design of an MRI-compatible biopsy needle with embedded fibers for real-time

shape detection.

The sensorized needle that we have developed is an early step towards increased

integration of force and deflection sensing for MRI-compatible medical devices. We

believe that it has the potential for more accurate MRI interventions, with a reduced

need to cycle the patient into and out of the MRI machine. In the future we envision

an MRI-compatible haptic master/slave device, allowing the interventionalist to

stand outside the MRI machine bore while manipulating the needle using deflection

and force information transmitted via a combination of visual and haptic displays.

FBG sensors are flexible, small and light, making them ideal for integration in

minimally invasive devices such as needles, probes and catheters.


5.2 System Modeling

We employed various mathematical methods to model our system and estimate

deflections and bend shapes of the needle using strain information, such as, beam

theory [93], parametric spline curves [31], and least energy curves [42]. Considering

the dimensions and mechanical properties of the needles we are prototyping, which

are relatively stiff, we can make the following assumptions during the interventional

operation:

1. The needle experiences negligible torsional loading along the needle length

axis.

2. The tip deflection is relatively small – less than 10% of the needle length – so

that small-strain linear beam theory applies.

3. The needle is sufficiently stiff that the bent profile is not complex; there are

at most one or two points of inflection.

Based on the above assumptions, we decided to use beam theory [93] for deflec-

tion estimation since beam theory is valid for a long, rigid, uniform body that is

fixed at one end, and can be simplified for systems that experience small deflections

and no torsions.

The needle can be modeled as a cantilever beam consisting of one fixed and

one free end. According to beam theory, since the local curvatures of the beam are

linearly proportional to the strains at the same locations, we can find local curvatures

of the beam with strain information obtained from the FBG sensors in our prototype.

If we use only two sensors, as shown in Figure 5.1, we know the curvatures at the

two sensor locations, and given the boundary conditions, we can construct a second

order polynomial curve fit which represents the curvature function of the needle.

The integral of the curvature function gives the slope function, and the integral of

the slope function yields the deflection function along the needle length, as shown


Table 5.1: Boundary conditions used for needle deflection estimation based on beamtheory.

Assumption Boundary Conditionsa

The deflection at the base is zero. Dxy(0) = Dyz(0) = 0

The slope at the base is zero. θxy(0) = θyz(0) = 0

The curvature at the tip is zero. gxy(L) = gyz(L) = 0

aDxy(y) and Dyz(y) are the deflection functions, θxy(y) and θyz(y) are the slope functions, andgxy(y) and gyz(y) are the curvature functions, in xy and yz planes, respectively. L is the length ofthe needle.

in Figure 5.1. The three boundary conditions used in our model are summarized in

Table 5.1.

We can obtain local curvatures using strain information obtained from the sensor

signals. Since the Bragg wavelength λB is

λB = 2neffΛ (5.1)

where neff is effective refractive index and Λ is the period of the grating, the re-

lationship between the wavelength shift, ∆λB, and the strain, ε, can be expressed

by∆λBλB

= (1− Pε) ε (5.2)

where Pε is photoelastic coefficient of the optical fiber.

For a cylindrical rod under pure bending [30],

ε =d

ρ= d · C, (5.3)

where d is the distance to the neutral plane, ρ is the radius of the curvature, and C

is the relative curvature. Finally, we have the relationship between the wavelength


Curvature (1/ρ)

Slope

Deflection

y

y

y

εy

dLocal curvature =

1

ρ=

y2y1

g(y) = ay2+by+c

(estimated curvature function)

Sensor 1 Sensor 2 εy: strain measured by FBG sensor

ρ: radius of curvature

d: distance from neutral axis

Slope:

Deflection: D(y) = ∫∫ g(y) dy

θ(y) = ∫ g(y) dy

z

x

y

0

0

0

Figure 5.1: Needle deflection estimation model using two strain sensors in 2D basedon beam theory.

shift and the curvature

∆λB = (1− Pε)λBε = (1− Pε) · 2neffΛ · d · C. (5.4)

Since Pε, neff , Λ, and d are all constants, the wavelength shift, ∆λB, is linearly

proportional to the curvature, C. Therefore, we can directly derive mappings for

the curvatures and wavelength shifts. Given the amounts of known curvatures, a

polynomial fit function can be derived. The second integral of the curvature function

with the above boundary conditions gives the estimated deflection equation as shown

in Figure 5.1.


5.3 Prototype Design

An FDA-approved 18 ga × 15 cm MRI-compatible histology biopsy needle (model

number: MR1815, E-Z-EM Inc, Westbury, NY) was selected as a basis for proto-

typing. 18 ga is a typical size used for MRI-guided interventions such as breast and

prostate biopsies. This needle is composed of two parts, a solid inner needle (stylet),

and a hollow outer needle (sheath). The outer needle is a thin walled tube made of

nonmagnetic nickel-cobalt-chromium-molybdenum alloy (MP35N). The inner stylet

has a core of nickel-chromium-molybdenum alloy (Inconel 625). The stylet initially

stays inside the outer sheath to prevent unwanted tissues or fluids from flowing into

the bore, and to stiffen the needle during insertion. When the needle tip reaches

the target tissues, the inner stylet is removed, and a syringe or other extraction

mechanism is connected to the base of the sheath to remove the targeted cells.

We incorporated optical fibers in the inner stylet, rather than the outer sheath,

for two reasons: the wall of the outer needle is too thin to modify for sensorizing

with optical fibers; and connecting the optical fiber cables at the stylet base does

not interfere with tissue extraction as it is removed prior to attaching a syringe or

other tool.

5.3.1 Sensor Configuration

Accurate estimation for the tip deflection of the MRI-compatible needle is dependent

on proper sensor placement, since the sensors measure strains only at the specific

locations of the needle. Due to the dimensions and stiffness of the needles used, it

was ascertained that there would likely be no more than two inflection points along

the needle [26]. Therefore, at least two sensor locations are required to estimate the

final tip position and to find a reasonable approximation of the bending profile of

the needle.

The two sensor locations should be determined in optimal positions in order

to obtain as much information as possible on tip deflections of the needle. The


locations of FBGs can be determined where the tip deflection errors and sensitivity

of the errors are minimized based on beam theory. Since the 3D needle deflection

can be always decomposed to two orthogonal planes, we decided to find the optimal

sensor locations in 2D first, and extend the number of sensors in a different plane

for 3D estimation.

There are various complicated loading conditions possible during real interven-

tional procedures. However, since in one plane, the actual needle deflection can

be simplified to either simple one-directional bending or an S-shape two-directional

bending, in our model, we considered the following four loading conditions to deter-

mine the optimal sensor locations:

1. Two point loads: one at the tip and the other at the mid-point – Figure 5.2

(A).

2. One distributed load and one point load: One distributed load from the base

to the mid-point and one point load at the tip – Figure 5.2 (B).

3. One distributed load: One distributed from the mid-point to the tip – Figure

5.2 (C).

4. Two distributed loads: one from the base to the mid-point and the other from

the mid-point to the tip – Figure 5.2 (D).

Two point loads – Figure 5.2 (A)

Assuming a point load is applied at the tip and an additional point load is at the

mid-point of the needle, the first sensor set should be placed between the needle base

and the location of the first load, and the second sensor set should be between the

two loads, as shown in Figure 5.3. Then, we can construct a real deflection model

using a curvature diagram which can be directly converted from a bending moment

diagram as shown in Figure 5.4.


Figure 5.2: Four loading conditions considered for optimum sensor location deter-mination. (A) Two point loads. (B) One distributed load and one point load. (C)One point load and one distributed load. (D) Two distributed loads. (y1 and y2 aretwo sensor locations to be determined, and L is the length of the needle.)

Since there are only two point loads, the curvature diagram is composed of

two continuous piecewise linear equations. Their second integrals, which are two

continuous third order polynomial equations, construct the real deflection model.

The real tip deflection is

Dreal(L) = −(

5

48F1 +

1

3F2

)L3 · 1

EI, (5.5)

where E and I are Young’s modulus and second moment of inertia of the needle,

respectively.


Figure 5.3: Needle models with different loading conditions: (A) no load, (B) twopoint loads in the same direction, and (C) two point loads in opposite directions,where y1 and y2 are the first and second sensor locations, and F1 and F2 are themagnitudes of two point loads at the mid-point and the tip of the needle, respectively.

However, if we use two sets of sensors, we only know the local curvatures at

the two sensor locations, and we can construct the estimated curvature function

using a second order polynomial curve fit. Then, the estimated deflection model

becomes the second integral of the estimated curvature function, which is a fourth

order polynomial function, and the estimated tip deflection is

Destimated(L) =1

12aL4 +

1

6bL3 +

1

2cL2, (5.6)

where ay21 + by1 + c = (F1 + F2)y1 −

(12F1 + F2

)L,

ay22 + by2 + c = F2y2 − F2L,

and aL2 + bL+ c = 0.

Next, the tip deflection error can be expressed as a function of two sensor loca-

tions as:


F1

Sh

ear

forc

e

0 50 100 150Needle length (y) [mm]



Cu

rva

ture

(1

/ρ)

Slo

pe

De

flec

tio

n

F1+F2

F2

L

LFFyFFyf )2

1()()(

21211+−+=

LFyFyf 222 )( −=

cbyayyg ++=2

)(

L/2

∫∫ dyyg )(

∫ dyyf )(1

∫ dyyg )(

dyyf∫∫ )(1

∫ dyyf )(2

dyyf∫∫ )(2

F2

y

Real model

Estimated model0

0

0

0

Figure 5.4: Example of deflection estimation process based on beam theory. Aneedle was fixed at the base (y = 0 mm), and two point loads of -0.6 N and 0.2 Nwere applied at the mid-point (y = 75 mm) and at the tip (y = 150 mm) of theneedle, respectively.


E(y1, y2) = Dreal(L)−Destimated(L)

= −(

5

48F1 +

1

3F2

)L3 · 1

EI−(

1

12aL4 +

1

6bL3 +

1

2cL2

). (5.7)

Since we are only interested in minimizing the magnitude of the error, we finally

have a simple two-dimensional nonlinear optimization problem such as,

Minimize |E(y1, y2)| =∣∣∣∣−( 5

48F1 +

1

3F2

)L3 · 1

EI−(

1

12aL4 +

1

6bL3 +

1

2cL2

)∣∣∣∣(5.8)

Subject to 0 < y1 ≤ 2L

,

2L≤ y2 < L,

F1, F2, L, E, and I are constants.

To solve this problem the tip position error was plotted with all possible sensor

locations, and two minimum error curves, on which the tip position error becomes

zero, were found as shown in Figure 5.5. In addition to the deflection error, the

sensitivity of deflection error to sensor placement needs to be checked to select the

sensor locations where the sensor placement error less affects the tip deflection error.

Since the deflection error sensitivity is much smaller in the left-lower curve than in

the right-upper curve from the 3D error plot, Figure 5.5 (A), the sensor locations

can be chosen from the left-lower curver in Figure 5.5 (B).

One distributed load and one point load – Figure 5.2 (B)

Another loading condition we considered is one distributed load from the base to

the mid-point and one point load at the tip. In the same way using beam theory, we

can find the deflection error plots, as shown in Figure 5.6. The deflection error plot

shows two minimum error curves again, and the left curve shown in the 2D plot,

Figure 5.6 (B), gives much less deflection error sensitivity to the sensor placement.


Figure 5.5: Tip position error plots for all possible sensor locations with two pointloads. (A) 3D contour plot. y1 and y2 represent the first and second sensor locations,respectively. (B) 2D projection of the 3D contour plot. Two white curves showpossible sensor locations where the tip position error is minimized.

One distributed load – Figure 5.2 (C)

The third loading condition taken into account is one distributed load from the

mid-point to the tip. In this case, only one minimum error curve was found in the

deflection error plot, as shown in Figure 5.7.

Two distributed loads – Figure 5.2 (D)

The last loading condition we considered is two distributed loads, one from the base

to the mid-point and the other from the mid-point to the tip. In this case, only one

minimum error curve appears in the deflection error plot, as shown in Figure 5.7.

By superimposing the plots, as shown in Figure 5.9, we can find the optimal

sensor locations which minimize the deflection error in all loading conditions. The

four curves do not come to a perfect intersection, yet have a small area in which all

the curves cross. Using this result, optimal sensor locations for the two sensors were

placed at 22 mm and 85 mm from the base of the needle.


Figure 5.6: Tip position error plots for all possible sensor locations with one dis-tributed and one point loads. (A) 3D contour plot. y1 and y2 represent the firstand second sensor locations, respectively. (B) 2D projection of the 3D contour plot.Two white curves show possible sensor locations where the tip position error isminimized.

0

20

4060

80

100

120

140

0

0.5

1

1.5

2

x1x2

0 10 20 30 40 50 60 70

80

90

100

110

120

130

140

150

x1

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

y1 [mm]

y2

[mm

]

y1 [mm]y2 [mm]

De

flec

tio

n e

rro

r [m

m]

(A) (B)

Figure 5.7: Tip position error plots for all possible sensor locations with one dis-tributed load. (A) 3D contour plot. y1 and y2 represent the first and second sensorlocations, respectively. (B) 2D projection of the 3D contour plot. The white curveshows possible sensor locations where the tip position error is minimized.


Figure 5.8: Tip position error plots for all possible sensor locations with two dis-tributed loads. (A) 3D contour plot. y1 and y2 represent the first and second sensorlocations, respectively. (B) 2D projection of the 3D contour plot. The white curveshows possible sensor locations where the tip position error is minimized.

0 10 20 30 40 50 60 70

80

90

100

110

120

130

140

150

Optimal sensor locations

(y1 = 22 mm, y2 = 85 mm)

y2

[mm

]

y1 [mm]

Two point loadsOne distributed and one point (tip) loadsOne point (tip) and one distributed loadsTwo distributed loads

Figure 5.9: Superimposed plot of all three simulation results, Figures 5.5, 5.6, 5.8,showing optimal sensor sensor locations: y1=22 mm and y2=85 mm from the baseof the needle.


5.3.2 Inner Stylet Design

The modified stylet has three grooves along the needle axis with 120◦ intervals,

and three optical fibers are attached inside the grooves. The minimum number of

strain sensors required to estimate local curvatures at a particular location along the

needle, is two: one to measure bending in the xy plane and the other for the yz plane

(see Figure 5.10). However, it is necessary to provide temperature compensation

because FBG sensors also have high sensitivity to changes in temperature [68]. In

the present case, since the diameter of the innter stylet is small (≈ 0.97 mm), we can

assume that the temperature is uniform across the needle diameter. If we incorporate

three FBGs at one location along the needle, we have one redundant sensor reading

that we can use for temperature compensation. In addition, since each optical fiber

contains two FBG sensors for strain measurement at two locations as discussed in

the previous section, there are a total of six FBG sensors in our prototype (two sets

of three sensors).

Figure 5.10 shows the design of the modified inner stylet incorporating optical

fibers. The stylet has three lengthwise grooves, 350 µm wide, separated at 120◦

intervals. Each groove holds an optical fiber with two FBG sensors at 22 mm and

85 mm from the base of the needle.

5.4 Needle Fabrication

We manufactured the prototype by making grooves via electrical discharge machin-

ing (EDM) along the stylet. EDM is ideal for this application, as it can create very

small features with high accuracy in any metal, and does not run the risk of intro-

ducing ferromagnetic particles. To ensure accuracy, and to simplify the process of

preparing multiple needles, we used a custom clamping fixture, which is also made

by wire EDM. The stylet is placed in the central 1.02 mm bore and clamped in place

using several set screws. EDM wire is threaded into each of the larger 3 mm holes


Figure 5.10: Prototype design with modified inner stylet incorporated with threeoptical fibers. Three identical grooves with 120◦ intervals are made on the innerstylet to embed optical fibers with FBGs along the needle length. (A) Midpointcross-section. (B) Magnified view of an actual groove. (C) Tip of the stylet.

for cutting the corresponding groove. In the future, if it becomes desirable to work

with smaller needles, optical fibers as thin as 40 µm diameter and correspondingly

thinner EDM wires can be used.

A magnified view of a single groove is shown in the inset in Figure 5.10 (C).

Optical fibers with an outer diameter of 350 µm were bonded in these grooves

using a low-viscosity bio-compatible cyanoacrylate adhesive. Since the contact area

between an optical fiber and the groove surface is large, we expect good shear

transfer between the needle and fibers.

Because EDM machining cannot cut plastic parts, we first removed the plas-

tic handle from the stylet using a heat gun and reattached it using epoxy after


machining. We also prepared holes in the plastic base, through which the optical

fibers could exit. The fibers protruding from the base were jacketed for increased

durability along their runs back to the optical interrogator.

5.5 Sensor Calibration

The needle prototype was calibrated for three-dimensional bending using two digital

cameras. Figure 5.11 (A) shows the calibration setup with orthogonally placed

cameras, and various tip deflections were made while images were taken in the xy

and yz planes, as shown in Figure 5.11 (B). Then, the images were processed using

the OpenCV [12] library to obtain the profile of the centerline of the needle. The

resolution of the digital imaging system used for the calibration was 0.05 mm/pixel.

Three separate experiments were conducted for the calibration as follows:

1. Only vertical (z-axis) tip loads were applied.

2. Only horizontal (x-axis) tip loads were applied.

3. Only temperature was changed; The temperature at each sensor location was

increased approximately from 20◦C to 55◦C and decreased back to 20◦C. Dur-

ing this temperature increase and decrease period, the real temperature was

measured using a digital temperature probe, and no mechanical strain was

applied to the needle.

5.5.1 Wavelength Shift vs. Curvature

Since we already know that FBG wavelength shift is linearly proportional to curva-

ture at the sensor location as discussed in Section 5.2, we can directly find a linear

mapping between the two variables. Figures 5.12, 5.13, and 5.14 show the calibra-

tion results from experiments 1, 2, and 3, respectively. All six FBG sensors provided

linear and consistent signals.


(A) (B)

Figure 5.11: (A) Three-dimensional calibration setup with orthogonally placed cam-eras and a light box. (B) Close up of the needle fixed to the calibration apparatus.The tip was deflected in the x and z directions.

Based on the calibration results, we can find a corresponding calibration matrix,

Cn, at the sensor location n, as following, when yn and sn are measured reference

and sensor signal at the sensor location n, respectively:

δyn = δsn ·Cn (5.9)

where δyn =[kxy kyz ∆t

]and δsn =

[∆λ1 ∆λ2 ∆λ3

]. kxy and kyz are

local curvatures in xy and yz planes, ∆t is the temperature change measured for

temperature compensation, and ∆λ1, ∆λ2, and ∆λ3 are wavelength shifts from the

three FBGs, respectively.

A simple way to solve for the Cn is using the Moore-Penrose pseudo inverse:

Cn = [δsTδs]−1δsT · δyn (5.10)


-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

x 10-3

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

-1 -0.5 0 0.5 1

x 10-3

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

Wavele

ngth

shift [n

m]

Curvature (1/ρ) [mm]

Wavele

ngth

shift [n

m]


Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Figure 5.12: Wavelength shifts measured in experiment 1 (x-axis loading).

and the tip deflection error is:

e = δsnCn − δyn. (5.11)

However, the accuracy levels of curvature measurement and temperature change

are different, and we have to normalize the error level using a weighting matrix [40].

The normalized error, e, can be written as

e = Ge = GδsnCn −Gδyn (5.12)

where G is a scaling matrix [55]. Then, we minimize the normalized error

eTe = eTGTGe = eTWe (5.13)

where W is a diagonal weight matrix. Finally, we can find weighted least squares

solution for the calibration matrix as follows:

Cn = [δsTWδs]−1δsTW · δyn (5.14)


-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

x 10-3

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

-1 -0.5 0 0.5 1

x 10-3

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

Wavele

ngth

shift [n

m]


Wavele

ngth

shift [n

m]


Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Figure 5.13: Wavelength shifts measured in experiment 2 (z-axis loading).

20 25 30 35 40 45 50 55 601530

1531

1532

1533

1534

1535

1536

1537

1538

1539

1540

20 25 30 35 40 45 50 551546

1547

1548

1549

1550

1551

1552

1553

1554

1555

1556

Wavele

ngth

[nm

]

Temperature [ºC]

Wavele

ngth

[nm

]

Temperature [ºC]

Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Figure 5.14: Wavelength measured in experiment 3 (temperature change).

Since we have two sensor locations (22 mm and 85 mm from the base of the

needle), a calibration matrix was found at each sensor location as:

C1 =

0.941 −3.599 3.061

−1.025 0.179 2.046

−2.844 −4.254 3.211

× 10−3 (5.15)


Table 5.2: Deflection error comparison for different deflection ranges using curvaturecalibration method.

Deflection Range (mm) -7≤ d ≤7 -10≤ d ≤10 -15≤ d ≤15

RMS of exy 0.35 0.38 0.38

RMS of eyz 0.26 0.26 0.28

and

C2 =

1.601 −0.509 1.451

−0.526 1.842 0.935

−1.516 −1.273 1.158

× 10−3. (5.16)

Using the calibration matrix for each sensor location, we can find local curvatures

from real-time sensor signals during the procedure. From the curvature measure-

ments we then estimate the deflection profile using beam theory, as shown in Figure

5.1, and extrapolate the needle tip position and bend profile. The calibration results

yielded the RMS values of deflection errors of 0.38 mm and 0.28 mm in the xy and

yz planes, respectively when the actual deflections were in the range of ± 15 mm.

Table 5.2 summarizes the RMS values of the deflection errors for different deflection

ranges.

The main error in estimating tip deflections with this method comes from inac-

curate modeling of the curvature function of the needle. Although there are various

loading conditions possible during actual interventional procedurs, we are construct-

ing the curvature function using a simple second order polynomial fit that does not

represent all kinds of loading conditions. Therefore, if we can collect more infor-

mation on loading conditions, we may be able to improve the accuracy of the tip

position estimation.

Another error source is inaccurate sensor placement during prototype fabrica-

tion. Since the optical fibers with FBGs were manually attached inside the machined


Figure 5.15: Assumptions for sensor location error calibration

grooves of the needle, it was difficult to place the sensors at the exact sensor loca-

tions. Consequently, we require another step to calibrate these manufacturing errors

for improved accuracy.

Before including the error terms in our model, we made two assumptions as

follows:

• Even though there are sensor location errors, all three sensors are placed at

the same location.

• The distance between two sensor locations (location 1 and 2) is always the

same in all three fibers.

Figure 5.15 summarizes these two assumptions. These two assumptions simplify

the problem by introducing only one error term ∆y in our calculation.


-3 -2 -1 0 1 2 30.26

0.28

0.3

0.32

0.34

0.36

0.38

0.4

0.42

0.44

delta y

rms

-3 -2 -1 0 1 2 30.45

0.46

0.47

0.48

0.49

0.5

0.51

0.52

0.53

0.54

0.55

RM

S v

alu

e o

f deflection e

rror

[mm

]

Sensor location error (∆y) [mm]

Norm

of R

MS

valu

es [m

m]

Sensor location error (∆y) [mm]

(A) (B)

xy-plane

yz-plane

Minimum error at ∆y = -0.91

Figure 5.16: Manufacturing error calibration result. (A) RMS value plot of tipdeflection error in two orthogonal planes, xy and yz. (B) Norm of RMS values oftip deflection errors.

Figure 5.16 is the experimental result of manufacturing error calibration show-

ing the tip deflection errors for different ∆y. We can find the value of ∆y which

minimizes the deflection error. The result shows the sensor locational error, ∆y, is

0.91 mm in our prtotype. Since the error was relatively small and the tip deflection

was not very sensitive to the manufacturing error in our prototype, the error cali-

bration did not contribute to reducing the deflection error significantly. However,

if the manufacturing error becomes larger or the estimation function is sensitive to

the error, this error calibration method will be useful.

5.5.2 Wavelength Shift vs. Deflection

If we can assume that there is a linear relationship between stimuli (applied tip

deflections) and the sensor signals (FBG wavelength shifts), we can derive a direct

linear mapping for tip deflection and sensor signal. The advantage of this calibration

method is that we can reduce the estimation errors caused by inaccurate modeling,

which we discussed in the previous section, since the manufacturing error is implicitly

calibrated. Figure 5.17 shows that the wavelength shifts of all six FBG sensors are


-20 -15 -10 -5 0 5 10 15 20-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

-20 -15 -10 -5 0 5 10 15 20-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Sensor 1

Sensor 2

Sensor 3

Sensor 4

Sensor 5

Sensor 6

Tip deflection [mm]

RM

S v

alu

e o

f deflection e

rror

[mm

]

Tip deflection [mm]

RM

S v

alu

e o

f deflection e

rror

[mm

]

(A)

(B)

Figure 5.17: Calibration result of all six FBG sensors for tip deflection. (A) Resultof Experiment 1 – deflection in yz plane. (B) Result of Experiment 2 – deflectionin xy plane.

linearly proportional to the tip deflection in the range of ± 15 mm.

Based on the calibration result in Figure 5.17, we can find a calibration matrix

for the mapping between wavelength shift and tip deflection as follows:


δy = δs ·C (5.17)

where

δy =[δyL/2,xy δyL/2,yz δyL,xy δyL,yz ∆t1 ∆t2

](5.18)

and

δs =[

∆λ1 ∆λ2 ∆λ3 ∆λ4 ∆λ5 ∆λ6

]. (5.19)

δyL/2,xy and δyL/2,yz, are midpoint deflections and δyL,xy and δyL,yz are tip deflec-

tions in xy and yz planes, respectively, ∆t is temperature change with respect to

the nominal temperature, C is the calibration matrix, ∆λ1, ∆λ2, and ∆λ3 are wave-

length shifts of the FBGs at the first sensor location, and ∆λ4, ∆λ5, and ∆λ6 are

wavelength shifts at the second location. Although we are mostly interested in

finding the tip deflections, δyL,xy and δyL,yz, the midpoint deflections, δyL/2,xy and

δyL/2,yz, are useful for estimating the bend profile of the needle.

As before, we can solve for the calibration matrix using the Moore-Penrose pseu-

doinverse. We also applied a weighting matrix to normalize midpoint deflection,

endpoint deflection and temperature variations following the method of [40], as dis-

cussed in Section 5.5.1. The calculated calibration matrix is:

C =

0.8262 2.4886 −3.4845 2.4031 22.0231 5.3903

2.4595 −2.3734 4.2011 −4.6952 16.4660 −3.0896

−3.3192 −0.1909 −0.6269 2.2194 18.6473 −2.4679

5.1925 −7.8612 31.9975 −12.5884 −9.3745 4.0978

−8.0867 13.6105 −16.0625 37.9421 −8.4017 14.0021

−4.2215 −8.0398 −23.3003 −25.8968 −0.0750 13.5217

(5.20)

.

Our calibration results yielded the RMS values of deflection errors of 0.11 mm

and 0.07 mm in the xy and yz planes, respectively when actual deflections were in

the range of ±15 mm. Table 5.3 summarizes the RMS values of the deflection errors


Table 5.3: Deflection error comparisons for different deflection ranges with directdeflection estimation.

Deflection Range (mm) -7≤ d ≤7 -10≤ d ≤10 -15≤ d ≤15

RMS of eL/2,xy 0.11 0.11 0.12

RMS of eL/2,yz 0.05 0.06 0.06

RMS of eL,xy 0.07 0.09 0.11

RMS of eL,yz 0.07 0.08 0.07

for different deflection ranges.

5.6 System Integration

Using the calibration described in the previous section, we developed a real-time

needle deflection and bend shape monitoring system. The three optical fibers, each

containing two FBG sensors, were combined and routed to a diffraction grating

based FBG interrogator (D*Sense 1400, IFOS, Santa Clara, CA). Although optical

power is attenuated by 20-40 % using this approach, the signals were strong enough

to read over a few meters of fiber. The update rate for this system was 4 Hz, limited

by the sampling rate of the interrogator. The interrogator readings were transmitted

to a laptop computer over a USB connection.

The sensor signals were processed using LabVIEW (National Instruments, Austin,

TX) for data acquisition and MATLAB (MathWorks, Inc., Natick, MA) for post-

processing and display. The peak wavelength values are obtained in LabVIEW dy-

namically from Dynamic-link library (DLL) files provided in the embedded firmware

of the interrogator. Then, the dynamic peak wavelength values are passed to a cus-

tom MATLAB script which includes information on the calibration matrix calcula-

tion for the tip deflection and bend shape estimation. Finally, the MATLAB script

generates a graphical model of the needle which shows the current needle shape on


Figure 5.18: Screen capture of the display of the real-time monitoring system.

a computer screen. Figure 5.18 shows a screen capture of the display.

Magnetic susceptibility and safety are concerns involving surgical tools and de-

vices used in the MRI field [80]. Consequently, the interrogator and laptop computer

were located remote from the MRI machine. The fiber optic cables are non-metallic

and do not interfere with the MRI magnetic field or machinery.


5.7 Preliminary MRI Scanner Tests

The FBG needle prototype, which can potentially be used with various interven-

tional MR-guided procedures, was tested for MRI compatibility in order to prove:

1. No imaging artifacts are caused by the sensors.

2. The sensor signals are not affected by the MRI scanner.

The modified needle presented the same degree of artifact as an unmodified

needle. The dark artifact which can be seen at the trocar tip (Figure 5.19 (A)),

is due to the intersection of the different materials present in the outer sheath and

inner stylet; it is equally seen in this imaging plane with the unmodified needle.

The second objective can be reached by comparing the estimated deflections

acquired by the sensor signals and the deflections measured in the MR images. The

needle was placed in a water bath and deflected with Nylon screws in five different

loading configurations, as shown in Figure 5.19. Results show the estimated tip

deflections are comparable to the deflections measured in the MR images.

5.8 Preliminary In-vivo Testing

The FBG needle prototype, which can potentially be used with various interven-

tional MR-guided procedures, was inserted into the prostate of a male beagle after

a cryroprobe was inserted. Wavelength readings were continuously measured at 4

Hz, and a series of MR images were taken. Later, the 2D “slices” were rendered

into a 3D image, from which 2D images along the plane of the needle were extracted

to compare with the position data calculated from the wavelength readings. Figure

5.20(A) shows reformatted coronal and sagittal MR-images of the prostate of the

test subject with the needle prototype inserted. The deflection and bend shape was

estimated using the FBG sensor signals and reconstructed graphically as shown in


50 100 150 200 250 300 350 400 450 500 550

0

5

10

15

TimeTime [sec]

Estim

ate

d d

eflection [m

m]

3.77 mm 6.91 mm 9.18 mm 10.85 mm 13.63 mm(A)

(B)

3.65 mm

6.84 mm

9.19 mm

10.75 mm

13.44 mm

Figure 5.19: (A) MRI scanned images with different deflections. The deflectionsrelative to MR images were found using a measurement tool in OsiriX [79] softwarefor viewing medical DICOM images. (B) Estimated needle deflections using FBGsensors.


(A) (B)

Figure 5.20: (A) 3T-MRI of the prototype needle in the prostate of dog in obliquecoronal (a, upper) and oblique sagittal (a, lower) views from reformatted 3D SPGRimages. (B) Optically measured 3D estimation of the needle shape. Axes scalesare exaggerated to highlight bending. Note the complete absence of any artifactsor interactions between the MRI and the optical shape-sensing methods despitesimultaneous scanning and optical FBG interrogation.

Figure 5.20(B). The estimation showed deflections of 2 mm and 2.5 mm in along

the x and z axes, respectively (scale exagerated to highlight flexing).


As an extension of the application of FBG sensors to robotic devices used in extreme

environments, a magnetic resonance imaging (MRI)–compatible biopsy needle was

developed, which was instrumented with optical fiber Bragg gratings (FBGs) for

measuring forces and bending deflections on the needle as it is inserted into tissues.

During procedures such as diagnostic biopsies and localized treatments, it is useful

to track any tool deviation from the planned trajectory to minimize positioning

error and procedural complications. The goal is to display tool deflections in real-

time, with greater bandwidth and accuracy than when viewing the tool in MR

images. A standard 18 ga (≈ 1 mm diameter) × 15 cm inner needle is prepared


using a custom-designed fixture, and 350 µm deep grooves are created along its

length. Optical fibers are embedded in the grooves. Two sets of sensors, located at

different points along the needle, provide a measurement of the tip force, and an

estimate of the bent profile, as well as temperature compensation. After calibration,

the measured tip position was accurate to within 0.1 mm. Tests of the needle in

a canine prostate showed that it produced no adverse imaging artifacts when used

with the MR scanner and no sensor signal degradation from the strong magnetic

field.

Chapter 6

Conclusions

In conclusion, this work has initiated the study of fiber optic force and position

sensing in compact and complicated robot structures and robotic tools used in ex-

treme environments. The work should not be considered as a simple replacement

for existing sensing technologies. It is a new method of robot sensing that extends

the potential area where robots could perform difficult and dangerous tasks as a

replacement of human beings by superseding the limitations of current technolo-

gies. The final chapter summarizes the work discussed in the preceding chapters

and presents suggestions for possible extensions and future work. A description of

the major contributions can be found in Section 1.3.

6.1 Summary of Results

The most significant result produced by this research is a new optical method of

accurate measurement of contact forces and deflections on robotic structures. Fiber

optic sensors were embedded in a polymer based robot end-effector, a robot finger,

to measure contact forces. The finger structure was miniaturized by embedding

bend-insensitive optical fibers. The application of FBG sensors was extended to

medical tools used for minimally invasive therapies in MRI environments.

88

CHAPTER 6. CONCLUSIONS 89

As a part of this progression of developing robotic structures with sensing, a

rapid prototyping process, Shape Deposition Manufacturing, was modified to sup-

port the fabrication of hollow, plastic mesh structures with embedded components.

The sensors were embedded near the base for high sensitivity to imposed loads.

The resulting structure is light and rugged. In initial experiments, the sensorized

structure demonstrated minimum detectable force changes of less than 0.02 N and

practical force measurement resolutions of less than 0.15 N, and a dominant fre-

quency at 167 Hz. With more precise location of the sensors, higher sensitivities

should be possible in the future. We also note that any frequency limit is provided

by the mechanical finger system, not the interrogator which can measure dynamic

strains to 5 kHz. A copper mesh in the structure reduces viscoelastic creep and

provides thermal shielding. A single FBG temperature compensation sensor at the

center of the hollow finger helps to reduce the overall sensitivity to thermal varia-

tions. However, the central sensor is sufficiently distant from the exterior sensors

that changes in temperature produce noticeable transient signals. This effect can

be reduced in the future by using a larger number of sensors and locating thermal

compensation sensors near the exterior of the structure, where they undergo the

same transient thermal strains as the other sensors.

Experiments were also conducted to investigate the finger prototype’s ability to

localize contact forces. Although the ability to localize forces with just four exte-

rior sensors is limited, the results show that the exoskeletal structure does respond

globally to point contacts in a predictable way. With a larger number of sensors,

more accurate contact localization will be possible. Increasing the number of sen-

sors is relatively straightforward, as multiple FBGs can be located along each fiber

with multiplexing. A robot hand with the finger prototypes was operated in a hy-

brid control scheme. The finger sensors are capable of resolving small forces and

are immune to electromagnetic disturbances so that the system can be mounted

on a large industrial robot, or in other applications where large magnetic fields are

present, without concern for shielding and grounding. For communication, it suffices


to route a single fiber down the robot arm.

A miniaturized force sensing robot finger at human-scale was designed and devel-

oped by embedding four FBG sensors inscribed in a 80 µm bend-insensitive optical

fiber. The miniaturized finger prototype showed capability of three-axis force sens-

ing. The resolutions of force measurement were 0.05 N in x and y axes and 0.16

N in z axis. Carbon fibers were embedded in addition to the optical fiber for re-

inforcement and to deal with creep effects of a polymer structure. The modified

SDM process successfully enabled the fabrication of human scale exoskeletal robot

fingertips. Future versions of this prototype will incorporate additional sensors for

thermal compensation and a modified design for greater sensitivity to axial loads.

As an extension of the application of FBG sensors to robotic devices used in ex-

treme environments, a magnetic resonance imaging (MRI)–compatible biopsy needle

was instrumented with optical fiber Bragg gratings (FBGs) for measuring bending

deflections on the needle as it is inserted into tissues. After calibration, the mea-

sured tip position was accurate to within 0.1 mm. Tests of the needle in a canine

prostate showed that it produced no adverse imaging artifacts when used with the

MR scanner and no sensor signal degradation from the strong magnetic field.

6.2 Future Work

Because of the novelty of using fiber optics for robot sensing, there is a wide range

of areas that can be explored to improve upon this work.

The first area can be extending the exoskeleton structure to the entire robot

limbs and body with composite materials and increased number of embedded op-

tical sensors. A composite robot will be light, with relatively high robustness, as

compared to conventional metal structured robots. Benefiting from the embedded

fiber optic sensors, the robot will be more autonomous and responsive to external

events, while having a more compact and simpler design. Also, the robot will be

fully immune to any electro-magnetic interferences.


Another area of work is minimally invasive surgeries and procedures in MRI-

environments. The field of minimally invasive medical procedures represents a new

application area for fiber Bragg grating sensors, albeit with significant design and

manufacturing challenges. Foremost among these is the need to minimize sensor

package size and to integrate sensors directly with surgical and diagnostic tooling.

Given that fiber-optic devices are inherently MRI-compatible, do not interact with

the MRI process, and do not cause significant imaging artifacts, they provide an

ideal method of sensing the configuration and forces upon interventional devices in

the MRI environment. We plan to further test this apparatus in MR environments

and to develop an API to overlay the real-time deflection results on MR images.

Interventional radiologists can subsequently use the deflection information to inter-

actively change the image scanning plane.

Ultimately, tools such as biopsy needles and cryoprobes will be manipulated

using an MRI-compatible master/slave device. Such devices enable the physician to

control tools within the narrow bore of an MRI scanner. In this case, it is desirable

to track the endpoint deflections of the tool in real-time to ensure that the desired

targets are reached. In addition, it may be advantageous to measure forces on the

tool, for example, to provide haptic feedback to the interventionalist about tissue

hardness and texture while palpating for tumors or assessing injury. It would be

desirable to build a sensorized haptic master/slave to further aid in probe insertion

and manipulation for prostate cryosurgery. Future work also includes a method to

actively steer the needle or probe tip.

Applications for FBGs and micro-scale fiber optics include incorporation into

existing MRI-compatible robots and apparatuses for needle or probe positioning

[15, 20, 27, 53, 54, 67]. Any needle-driving robot will experience the same deflections

that physicians encounter. FBG sensorized needles and probes can be used to help

plan the trajectories of these robots [14] to compensate for common deflections in

various tissues, as well as measure contact tissue forces. The modified stylet can also

be used as a force gauge to validate tissue deformation models in vivo for surgical


CAD/CAM procedures [48]. Because of the novelty of fiber optics for robot sensing,

there is a wide range of areas that can be explored.

Bibliography

[1] M. Amano, Y. Okabe, N. Takeda, and T. Ozaki. Structural health monitoring

of an advanced grid structure with embedded fiber Bragg grating sensors.

Structural Health Monitoring, 6(4):309–324, 2007.

[2] R. O. Ambrose, H. Aldridge, R. S. Askew, R. R. Burridge, W. Bluethmann,

M. Diftler, C. Lovchik, D. Magruder, and F. Rehnmark. Robonaut: NASA’s

space humanoid. IEEE Intelligent Systems and their Applications, 15(4):57–

63, 2000.

[3] L. Ascari, P. Corradi, L. Beccai, and C. Laschi. A miniaturized and flexi-

ble optoelectronic sensing system for a tactile skin. International Journal of

Micromechanics and Microengineering, 17:2288–2298, 2007.

[4] S. A. Bailey, J. G. Cham, M. R. Cutkosky, and R. J. Full. Biomimetic robotic

mechanisms via shape deposition manufacturing. 9th International Symposium

of Robotics Research, pages 321–327, 1999.

[5] F. G. Barth. Spider mechanoreceptors. Current Opinion in Neurobiology 2004,

14:415–422, 2004.

[6] F. G. Barth and J. Stagl. The slit sense organs of arachnids. Zoomorphologie,

86:1–23, 1976.

[7] A. Bicchi and G. Canepa. Optimal design of multivariate sensors. Measure-

ment Science and Technology, 5:319–332, 1994.

93

BIBLIOGRAPHY 94

[8] R. J. Black, D. Zare, L. Oblea, Y-L. Park, B. Moslehi, and C. Neslen. On

the gage factor for optical fiber grating strain gages. Conference Proceed-

ings of the Society for the Advancement of Materials and Process Engineering

(SAMPE’08), 53, 2008.

[9] R. Blickhan and F. G. Barth. Strains in the exoskeleton of spiders. Journal

of Comparative Physiology A, 157:115–147, 1985.

[10] W. Bluethmann, R. Ambrose, M. Diftler, S. Askew, E. Huber, M. Goza,

F. Rehnmark, C. Lovchik, and D. Magruder. Robonaut: A Robot designed to

work with humans in space. Autonomous Robots, 14:179–197, 2003.

[11] P. Blumenfeld, N. Hata, S. DiMaio, K. Zou, S. Haker, G. Fichtinger, and

C. Tempany. Transperineal prostate biopsy under magnetic resonance image

guidance: A needle placement accuracy study. Journal of Magnetic Resonance

Imaging, 26:688694, 2007.

[12] G. R. Bradski and A. Kaehler. Learning OpenCV. O’Reilly, Sebastopol, CA,

1st edition, 2008.

[13] L. Carvalho, J. C. C. Silva, R. N. Nogueira, J. L. Pinto, H. J. Kalinowski, and

J. A. Simoes. Application of Bragg grating sensors in dental biomechanics.

The Journal of Strain Analysis for Engineering Design, 41(6):411–416, 2006.

[14] F. Cepolina and R. C. Michelini. Robots in medicine: A survey of in-body

nursing aids introductory overview and concept design hints. In Proceedings

of ISR 2004, 35 International Symposium on Robotics. ISR, 2004.

[15] K. Chinzei and K. Miller. Towards MRI guided surgical manipulator. Medical

science monitor : international medical journal of experimental and clinical

research, 7(1):153–163, 2001.

[16] L. A. Danisch, K. Englehart, and A. Trivett. Spatially continuous six degree of

freedom position and orientation sensor. Sensor Review, 19(2):106–112, 1999.

BIBLIOGRAPHY 95

[17] C. R. Dennison, P. M. Wild, M. F. Dvorak, D. R. Wilson, and P. A. Cripton.

Validation of a novel minimally invasive intervertebral disc pressure sensor

utilizing in-fiber Bragg gratings in a porcine model: An ex vivo study. Spine,

33(17):E589–E594, 2008.

[18] J. A. Derbyshire, R. S. Hinks, A. W. Graham, and R. M. Henkelman. Dynamic

scan plane tracking using MR position monitoring. U.S. Patent, 5,947,900,

1998.

[19] S. Dimaio, E. Samset, G. Fischer, I. Iordachita, G. Fichtinger, F. Jolesz, and

C. Tempany. Dynamic MRI scan plane control for passive tracking of instru-

ments and devices. Medical Image Computing and Computer-Aided Interven-

tion - MICCAI 2007, 10(Pt2):50–58, 2007.

[20] S. P. Dimaio, G. S. Fischer, S. J. Maker, N. Hata, I. Iordachita, C. M. Tem-

pany, R. Kikinis, and G. Fichtinger. A system for MRI-guided prostate in-

terventions. Proceedings of the first IEEE/RAS-EMBS International Confer-

ence on Biomedical Robotics and Biomechatronics (BioRob2006), pages 68–73,

2006.

[21] A. Dollar, C. R. Wagner, and R. D. Howe. Embedded sensors for

biomimetic robotics via shape deposition manufacturing. Proceedings of the

first IEEE/RAS-EMBS International Conference on Biomedical Robotics and

Biomechatronics (BioRob2006), 2006.

[22] C. L. Dumoulin, S. P. Souza, and R. D. Darrow. Real-time position moni-

toring of invasive devices using magnetic resonance. Magnetic Resonance in

Medicine, 29(3):411–415, 1993.

[23] W. Ecke, I. Latka, R. Willsch, Reutlinger A., and R. Graue. Fiber optic

sensor network for spacecraft health monitoring. Measurement Science and

Technology, 12(7):974–980, 2001.

BIBLIOGRAPHY 96

[24] ENDOSENSE. Medical apparatus system having optical fiber load sensing

capability, 2006.

[25] A. F. Fernandez, F. Berghmans, B. Brichard1, P. Megret, M. Decreton,

M. Blondel, and A. Delchambre. Multi-component force sensor based on mul-

tiplexed fibre Bragg grating strain sensors. Measurement Science and Tech-

nology, 12:810–813, 2001.

[26] D. G. Fertis. Advanced Mechanics of Structures. CRC Press, 1st edition, April

1996.

[27] G. S. Fischer, I. Iordachita, C. Csoma, J. Tokuda, S. P. Dimaio, C. M.

Tempany, N. Hata, and G. Fichtinger. Mri-compatible pneumatic robot for

transperineal prostate needle placement. Mechatronics, IEEE/ASME Trans-

actions on, 13(3):295–305, 2008.

[28] R. F. Foelix. Biology of Spiders. Oxford University Press US, 2nd edition,

1996.

[29] E. J. Friebele, C. G. Askins, A. B. Bosse, A. D. Kersey, H. J. Patrick, W. R.

Pogue, M. A. Putnam, W. R. Simon, F. A. Tasker, W. S. Vincent, and S. T.

Vohra. Optical fiber sensors for spacecraft applications. Smart Materials and

Structures, 8:813–838, 1999.

[30] J. M. Gere and B. J. Goodno. Mechanics of Materials. CL-Engineering, 7th

edition, April 2008.

[31] J. M. Glass. Smooth-curve interpolation: A generalized spline-fit procedure.

BIT Numerical Mathematics, 6(4):277–293, 1966.

[32] K. Grattan. Fiber optic sensor technology: an overview. Sensors and Actua-

tors A: Physical, 82(1-3):40–61, May 2000.

BIBLIOGRAPHY 97

[33] W. Griffin, W. M. Provancher, and M. R. Cutkosky. Feedback strategies

for telemanipulation with shared control of object handling forces. Presence:

Teleoperations and Virtual Environments, MIT Press, 14(6):720–731, 2005.

[34] T. Guo, Q. Zhao, H. Zhang, C. Zhang, G. Huang, L. Xue, and X. Dong.

Temperature-insensitive fiber Bragg grating dynamic pressure sensing system.

Optics Letters, 31(15):2269–2271, 2006.

[35] T. C. Hales. The honeycomb conjecture. 1999, Available:

http://arxiv.org/abs/math.MG/9906042, 1999.

[36] K. O. Hill and G. Meltz. Fiber Bragg grating technology fundamentals and

overview. Journal of Lightwave Technology, 15(8):1263–1276, 1997.

[37] N. Hogan. Impedance control: An approach to manipulation, part I, Theory.

ASME Journal of Dynamic Systems, Measurement, and Control, 107(1):1–7,

1985.

[38] N. Hogan. Impedance control: An approach to manipulation, part II, Imple-

mentation. ASME Journal of Dynamic Systems, Measurement, and Control,

107(1):8–16, 1985.

[39] N. Hogan. Impedance control: An approach to manipulation, part III, ap-

plication. ASME Journal of Dynamic Systems, Measurement, and Control,

107(1):17–24, 1985.

[40] J. M. Hollerbach and C. W. Wampler. The calibration index and taxonomy for

robot kinematic calibration methods. The International Journal of Robotics

Research, 15(6):573–591, December 1996.

[41] J. Hong and X. Tan. Calibrating a VPL dataglove for teleoperating the

utah/mit hand. Proceedings of the 1989 IEEE International Conference on

Robotics and Automation, 3:1752–1757, 1989.

BIBLIOGRAPHY 98

[42] B. K. P. Horn. The curve of least energy. ACM Transactions on Mathematical

Software, 9(4):441–460, 1983.

[43] S. G. Hushek, B. Fetics, R. M. Moser, N. F. Hoerter, L. J. Russel, A. Roth,

D. Polenur, and E. Nevo. Initial clinical experience with a passive electro-

magnetic 3D locator system. In Proceedings of the 5th Interventional MRI

Symposium, pages 73–74, 2004.

[44] Corning, Inc. Corning SMF-28e+TM optical fiber product information. Avail-

able: http://doclib.corning.com/OpticalFiber/pdf/pi1463.pdf, 2006.

[45] I. Iordachita, Z. Sun, and M. Balicki. A sub-millimetric, 0.25 mN resolution

fully integrated fiber-optic force-sensing tool for retinal microsurgery. Inter-

national Journal of Computer Assisted Radiology and Surgery, 4(4):383–390,

1989.

[46] J. Jung, H. Nam, B. Lee, J. O. Byun, and N. S. Kim. Fiber Bragg grating

temperature sensor with controllable sensitivity. Applied Optics, 38(13):2752–

2754, 1999.

[47] K. Kamiyama, H. Kajimoto, M. Inami, N. Kawakami, and S. Tachi. Devel-

opment of a vision-based tactile sensor. IEEE Transactions on Sensors and

Micromachines, 123(1):16–22, 2003.

[48] P. Kazanzides, G. Fichtinger, G. D. Hager, A. M. Okamura, L. L. Whit-

comb, and R. H. Taylor. Surgical and interventional robotics - core concepts,

technology, and design [Tutorial]. Robotics & Automation Magazine, IEEE,

15(2):122–130, 2008.

[49] H. Kazernooni, T. B. Sheridan, and P. K. Houpt. Robust compliant motion

for manipulators, parts I-II. IEEE Transactions of Robotics and Automation,

RA-2(2):83–105, 1986.

BIBLIOGRAPHY 99

[50] A. D. Kersey, M. A. Davis, H. J. Patrick, M. LeBlanc, K. P. Koo, C. G.

Askins, M. A. Putnam, and E. J. Friebele. Fiber grating sensors. Journal of

Lightwave Technology, 15(8):1442–1463, 1997.

[51] J. M. Ko and Y. Q. Ni. Technology developments in structural health moni-

toring of large-scale bridges. Engineering Structures, 27(12):1715–1725, 2005.

[52] E. Kochavi, D. Goldsher, and H. Azhari. Method for rapid MRI needle track-

ing. Magnetic Resonance in Medicine, 51(5):1083–1087, 2004.

[53] A. Krieger, G. Metzger, G. Fichtinger, E. Atalar, and L. L. Whitcomb. A

hybrid method for 6-dof tracking of MRI-compatible robotic interventional

devices. In Robotics and Automation, 2006. ICRA 2006. Proceedings 2006

IEEE International Conference on, pages 3844–3849, 2006.

[54] A. Krieger, R. C. Susil, C. Menard, J. A. Coleman, G. Fichtinger, E. Atalar,

and L. L. Whitcomb. Design of a novel MRI compatible manipulator for image

guided prostate interventions. Biomedical Engineering, IEEE Transactions on,

52(2):306–313, 2005.

[55] C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. Prentice-

Hall, Inc., Englewood Cliffs, NJ, 1974.

[56] D. A. Leung, J. F. Debatin, S. Wildermuth, G. C. Mckinnon, D. Holtz, C. L.

Dumoulin, R. D. Darrow, E. Hofmann, and G. K. von Schulthess. Intravas-

cular MR tracking catheter: preliminary experimental evaluation. Am. J.

Roentgenol., 164(5):1265–1270, May 1995.

[57] H-N. Li, D-S. Li, and G-B. Song. Recent applications of fiber optic sensors to

health monitoring in civil engineering. Engineering Structures, 26(11):1647–

1657, 2004.

BIBLIOGRAPHY 100

[58] G. Liu and Z. Li. A unified geometric approach to modeling and control of

constrained mechanical systems. IEEE Transactions on Robotics and Automa-

tion, 18(4):574–587, 2002.

[59] C. S. Lovchik and M. A. Diftler. The Robonaut hand: A dexterous robot hand

for space. Proceedings of the 1999 IEEE International Conference on Robotics

and Automation, pages 907–912, 1999.

[60] H. Maekawa, K. Tanie, and K. Komoriya. Tactile feedback for multifingered

dynamic grasping. IEEE Control Systems Magazine, 17(1):63–71, 1997.

[61] S. Magne, S. Rougeault, M. Vilela, and P. Ferdinand. State-of-strain evalua-

tion with fiber Bragg grating rosettes: application to discrimination between

strain and temperature effects in fiber sensors. Applied Optics, 36(36):9437–

9447, 1997.

[62] J. Mandal, S. Pal, T. Sun, K. T. V. Grattan, A. T. Augousti, and S. A. Wade.

Bragg grating-based fiber-optic laser probe for temperature sensing. IEEE

Photonics Technology Letters, 16(1):218–220, 2004.

[63] C. Menona, R. Brodieb, S. Cliftb, and J. F. V. Vincent. Concept design

of strain sensors inspired by campaniform sensilla. Acta Astronautica, 64(2-

3):176–182, 2009.

[64] A. Mita and I. Yokoi. Fiber Bragg grating accelerometer for buildings and civil

infrastructures. Proceedings of SPIE, Sensors for Infrastructure monitoring II,

4330:479–486, 2001.

[65] D. T. Moran, K. M. Chapman, and R. S. Ellis. The fine structure of cockroach

campaniform sensilla. The Journal of Cell Biology, 48:155–173, 1971.

[66] W. W. Morey, G. Meltz, and J. M. Weiss. Recent advances in fiber-grating

sensors for utility industry applications. Proceedings of SPIE, Self-Calibrated

Intelligent Optical Sensors and Systems, 2594:90–98, 1996.

BIBLIOGRAPHY 101

[67] M. Muntener, A. Patriciu, D. Petrisor, M. Schar, D. Ursu, D. Y. Song, and

D. Stoianovici. Transperineal prostate intervention: Robot for fully automated

MR imaging–system description and proof of principle in a canine model.

Radiology, 247(2):543–549, May 2008.

[68] A. Othonos and K. Kalli. Fiber Bragg Gratings: Fundamentals and Applica-

tions in Telecommunications and Sensing. Artech House Publishers, 1999.

[69] S. Pal, J. Mandal, T. Sun, K. T. V. Grattan, M. Fokine, F. Carlsson, P. Y.

Fonjallaz, S. A. Wade, and S. F. Collins. Characteristics of potential fibre

Bragg grating sensor-based devices at elevated temperatures. Measurement

Science and Technology, (14):1131–1136, 2003.

[70] Y-L. Park, K. Chau, R. J. Black, and M. R. Cutkosky. Force sensing robot fin-

gers using embedded fiber Bragg grating sensors and shape deposition manu-

facturing. Proceedings of the 2007 IEEE International Conference on Robotics

and Automation, pages 1510–1516, 2007.

[71] Y-L. Park, S. Elayaperumal, B. L. Daniel, E. Kaye, K. B. Pauly, R. J. Black,

and M. R. Cutkosky. MRI-compatible haptics: Feasibility of using optical fiber

Bragg grating strain-sensors to detect deflection of needles in an MRI environ-

ment. International Society for Magnetic Resonance in Medicine (ISMRM)

2008, 16th Scientific Meeting and Exhibition, 2008.

[72] Y-L. Park, S. Elayaperumal, S. Ryu, B.L. Daniel, R.J. Black, B. Moslehi, and

M.R. Cutkosky. MRI-compatible haptics: Strain sensing for real-time estima-

tion of three dimensional needle deflection in MRI environments. International

Society for Magnetic Resonance in Medicine (ISMRM) 2009, 17th Scientific

Meeting and Exhibition, 2009.

BIBLIOGRAPHY 102

[73] Y-L. Park, S. C. Ryu, R. J. Black, K. K. Chau, B. Moslehi, and M. R.

Cutkosky. Exoskeletal force-sensing end-effectors with embedded optical fiber-

Bragg-grating sensors. IEEE Transactions on Robotics, 25(6):1319–1331, 2009.

[74] Y-L. Park, S. C. Ryu, R. J. Black, B. Moslehi, and M. R. Cutkosky. Finger-

tip force control with embedded fiber Bragg grating sensors. Proceedings of

the 2008 IEEE International Conference on Robotics and Automation, pages

3431–3436, 2008.

[75] I. Peterson. The honeycomb conjecture. Science News, 156(4):60, 1999.

[76] M. Raibert and J. Craig. Hybrid position/force control of manipulators.

ASME Journal of Dynamic Systems, Measurement, and Control, 102(2):126–

133, 1981.

[77] Y. J. Rao. In-fibre Bragg grating sensors. Measurement Science and Technol-

ogy, 8(355-375), 1997.

[78] E. M. Reimer and L. A. Danisch. Pressure sensor based on illumination of a

deformable integrating cavity. U.S. Patents, 5 917 180, 1999.

[79] A. Rosset and J. Heuberger. OsiriX imaging software: Advanced open-source

PACS workstation DICOM viewer. http://www.osirix-viewer.com.

[80] J. F. Schenck. The role of magnetic susceptibility in magnetic resonance

imaging: MRI magnetic compatibility of the first and second kinds. Medical

Physics, 23(6):815–850, 1996.

[81] H. Seraji. A new class of nonlinear PID controllers with robotic applications.

Journal of Robotic Systems, 15(3):161–181, 1998.

[82] H. Seraji, D. Lim, and R. Steele. Experiments in contact control. Journal of

Robotic Systems, 13(2):53–73, 1996.

BIBLIOGRAPHY 103

[83] H. Seraji and R. Steele. Nonlinear contact control for space station dexterous

arms. Proceedings of the 1998 IEEE International Conference on Robotics and

Automation, 1:899–906, 1998.

[84] E-A. Seyfarth, W. Eckweiler, and K. Hammer. Proprioceptors and sen-

sory nerves in the legs of a spider, Cupiennius salei (Arachnida, Araneida).

Zoomorphologie, 105:190–196, 1985.

[85] S. G. Silverman, B. D. Collick, M. R. Figueira, R. Khorasani, D. F. Adams,

R. W. Newman, G. P. Topulos, and F. A. Jolesz. Interactive MR-guided

biopsy in an open-configuration MR imaging system. Radiology, 197(1):175–

181, October 1995.

[86] A. Skordos, P. H. Chan, J. F. V. Vincent, and G. Jeronimidis. A novel strain

sensor based on the campaniform sensillum of insects. Philosophical Trans-

actions of the Royal Society of London Series A - Mathematical Physical and

Engineering Sciences, 360:239–253, 2002.

[87] D. S. Smith. The fine structure of haltere sensilla in the blowfly Calliphora

erythrocephala (Meig.) with scanning electron microscopic observations on the

haltere surface. Tissue and Cell, 1:443–484, 1969.

[88] J. S. Son, M. R. Cutkosky, and R. D. Howe. Comparison of contact sensor

localization abilities during manipulation. Proceedings of 1995 IEEE/RSJ

International Conference on Intelligent Robots and Systems, 2:96–103, 1995.

[89] Z. Sun, M. Balicki, J. Kang, J. Handa, R. Taylor, and I. Iordachita. Develop-

ment and preliminary data of novel integrated optical micro-force sensing tools

for retinal microsurgery. Proceedings of 2009 IEEE International Conference

on Robotics and Automation, pages 1897–1902, 2009.

BIBLIOGRAPHY 104

[90] R. Suresh, S. C. Tjin, and S. Bhalla. Multi-component force measurement

using embedded fiber Bragg grating. Optics and Laser Technology, 41(4):431–

440, 2009.

[91] N. Takahashi, A. Hirose, and S. Takahashi. Underwater acoustic sensor with

fiber Bragg grating. Optical Review, 4(6):691–694, 1997.

[92] S. Theriault, K. O. Hill, F. Bilodeau, D. C. Johnson, J. Albert, G. Drouin, and

A. Beliveau. High-g accelerometer based on an in-fiber Bragg grating sensor.

Optical Review, 4(1A):145–147, 1997.

[93] S. Timoshenko and D. H. Young. Engineering Mechanics. McGraw-Hill, New

York, 4th edition, 1956.

[94] S. C. Tjin, J. Hao, Y-Z. Lam, Y. C. Ho, and Ng B. K. A pressure sensor using

fiber Bragg grating. Fiber and Integrated Optics, 20(1):59–69, 2001.

[95] U. Utzinger and R. R-K. Rebecca. Fiber optic probes for biomedical optical

spectroscopy. Journal of Biomedical Optics, 8(1):121–147, 2003.

[96] J. F. V. Vincent, S. E. Clift, and C. Menon. Biomimetics of campaniform

sensilla: Measuring strain from the deformation of holes. Journal of Bionic

Engineering, 4(2):63–76, 2007.

[97] R. M. Voyles, G. Fedder, and P. K. Khosla. Design of a modular tactile sensor

and actuatator based on an electrorheological gel. Proceedings of the 1996

IEEE International Conference on Robotics and Automation, 1:13–17, 1996.

[98] L. E. Weiss, R. Merz, F. B. Prinz, G. Neplotnik, P. Padmanabhan, L. Schultz,

and K. Ramaswami. Shape deposition manufacturing of heterogenous struc-

tures. Journal of Manufacturing Systems, 16(4):239–248, 1997.

BIBLIOGRAPHY 105

[99] M. G. Xu, J-L. Archambault, L. Reekie, and J. P. Dakin. Discrimination

between strain and temperature effects using dual-wavelength fibre grating

sensors. Electronics Letters, 30(13):1085–1087, 1994.

[100] M. G. Xu, L. Dong, L. Reekie, J. A. Tucknott, and J. L. Cruz. Temperature-

independent strain sensor using a chirped Bragg gratingin a tapered optical

fibre. Electronics Letters, 31(10):823–825, 1995.

[101] X. Yi, J. Qian, L. Shen, Y. Zhang, and Z. Zhang. An innovative 3D colonoscope

shape sensing sensor based on FBG sensor array. Proceedings of the 2007

International Conference on Information Acquisition, pages 227–232, 2007.

[102] G. Zeng and A. Hemami. An overview of robot force control. Robotica, 15:473–

482, 1997.

[103] Y. Zhan, H. Wu, Q. Yang, S. Xiang, and H. He. Fiber grating sensors for high-

temperature measurement. Optics and Lasers in Engineering, 46(4):349–354,

2008.

[104] L. Zhang, J. Qian, Y. Zhang, and L. Shen. On SDM/WDM FBG sensor net

for shape detection of endoscope. Proceedings of the 2005 IEEE International

Conference on Robotics and Automation, 4:1986–1991, 2005.

[105] W. Zhang, E. Li, J. Xi, J. Chicharo, and X. Dong. Novel temperature-

independent FBG-type force sensor. Measurement Science and Technology,

16:1600–1604, 2005.

EMBEDDED OPTICAL SENSING FOR ROBOTS IN EXTREME …ry064nm3195/YLP_thesis-augmented.pdffor certain...

Documents

Transcript of EMBEDDED OPTICAL SENSING FOR ROBOTS IN EXTREME …ry064nm3195/YLP_thesis-augmented.pdffor certain...