Automatic Detection and Segmentation of Robot-Assisted Surgical Motions presented by Henry C. Lin...

Automatic Detection and Segmentation of Robot-Assisted Surgical Motions

presented by

Henry C. Lin

Henry C. Lin, Dr. Izhak Shafran, Todd E. Murphy,

Dr. David D. Yuh, Dr. Allison M. Okamura,

Dr Gregory D. Hager

2MICCAI 2005

ERC-CISST Johns Hopkins University

AuthorsHenry LinPhD Student

Computer Science

Izhak ShafranResearch Scientist

ECE

Todd MurphyMS, 2004

Mechanical Engineering

David YuhSurgeon

Cardiac Surgery

Allison OkamuraAssistant Professor

Mechanical Engineering

Gregory HagerProfessor

Computer Science

3MICCAI 2005


Can we automatically detect and segment the surgical motions common in both videos?

Can we quantitatively and objectively determine which video is an expert surgeon and which is an intermediate surgeon?

Motivation

QuickTime™ and aCinepak decompressor

are needed to see this picture.







Expert Intermediate

4MICCAI 2005


Cartesian Position Plots - Left Manipulator

--- Pull suture with left hand

--- Move to middle with needle

Expert Surgeon - trial 4 Intermediate Surgeon - trial 22

5MICCAI 2005


Previous Work

• Darzi, et al. Imperial College Surgical Assessment Device

(ICSAD) quantified motion information by tracking electromagnetic markers on a trainee’s hands.

• Rosen, et al.Used force/torque data from laparoscopic

trainers to create a hidden Markov model task decomposition specific to each surgeon.

6MICCAI 2005


Goals

• Train LDA-based statistical models with labeled motion data of an expert surgeon and an intermediate surgeon.

• Be able to accurately parse unlabeled raw motion data into a labeled sequence of surgical gestures in an automatic and efficient way.

• Ultimately create evaluation metrics to benchmark surgical skill.

7MICCAI 2005


Corpus

78 motion variables acquired at 10Hz(we use 72 of them)

4-throw suturing task

15 expert trials12 intermediate trials

each trial roughly 60 seconds in length

8MICCAI 2005


Gesture Vocabulary1. Reach for needle

2. Position needle

3. Insert and push needle through tissue

6. Pull suture with left hand

4. Move to middle with needle(left hand)

8. Orient needle with both hands

7. Pull suture with right hand

5. Move to middle with needle(right hand)

9MICCAI 2005


APIsignals

X(1,t)

X(78,t)

X(t)Local

FeatureExtraction

L(t) FeatureNormalization

N(t)Linear

DiscriminantAnalysis

Y(t)Probabilistic

(Bayes)Classifier

P(Y(t)|C)

ProbabilisticModels for

Surgical Motions

C(t)

System Approach

APIsignals

X(1,t)

X(72,t)

X(t)Local

FeatureExtraction

10MICCAI 2005


Local Feature Extractions

X(kt)

+ + + +

X(kt-m+1) X(kt+m-1)

+ +

X(kt-m) X(kt+m)

€

L(kt ) = [X(kt−m ) | X(kt−m+1) | ... | X(kt ) | ... | X(kt+m−1) | X(kt+m )]

|L(kt)| = (2m+1)|X(kt)| Example: m=5, |L| = 792

11MICCAI 2005


APIsignals

X(1,t)

X(78,t)

X(t)Local

FeatureExtraction


N(t)Linear


Y(t)Probabilistic

(Bayes)Classifier

P(Y(t)|C)


Surgical Motions

C(t)

System Approach

APIsignals

X(1,t)

X(72,t)

X(t)Local

FeatureExtraction


N(t)Linear


12MICCAI 2005


Linear Discriminant Analysis

x1

x2

The objective of LDA is to perform dimensionality reduction while preserving as much of the class discriminatory information as possible.

13MICCAI 2005


where the linear transformation matrix W is estimated by maximizing the Fisher discriminant.

Linear Discriminant Analysis

€

Y (k) =W (N(k))

Fisher discriminant - ratio of distance between the classes and the average variance of each class

LDA

class-labeledmotion data

expected reducedoutput dimension

reduceddimension

motion data

14MICCAI 2005


LDA Reduction (6 Labeled Classes, 3 Dimensions)Expert Surgeon

15MICCAI 2005


LDA Reduction (6 Labeled Classes, 3 Dimensions)Intermediate Surgeon

16MICCAI 2005


Storage Savings of LDA

Method Temporal Neighbors (m) Space required (values)

Raw no - 432,000

Raw + temporal

yes 5 4,752,000

LDA-based yes 5 18,000

For a 10 minute procedure (6000 input samples)

17MICCAI 2005


APIsignals

X(1,t)

X(78,t)

X(t)Local

FeatureExtraction


N(t)Linear


Y(t)Probabilistic

(Bayes)Classifier

P(Y(t)|C)


Surgical Motions

C(t)

System Approach

APIsignals

X(1,t)

X(78,t)

X(t)Local

FeatureExtraction


N(t)Linear


Y(t)Probabilistic

(Bayes)Classifier

P(Y(t)|C)


Surgical Motions

C(t)

18MICCAI 2005


Results

• ‘Leave 2 out’ cross-validation paradigm used. 15 expert trials, 15 rounds

• Output of 2 test trials were compared against the manually labeled data.

• The average across the 15 tests was used to measure performance.

Training set (13) Test set (2)

19MICCAI 2005


Results

20MICCAI 2005


Resultsn Number of

labeled classesLDA output dimensions

% correct

1 6 3 91.26

2 6 4 91.46

3 6 5 91.14

4 5 3 91.06

5 5 4 91.34

6 5 3 92.09

7 5 4 91.92

8 4 3 91.88

21MICCAI 2005


Contributions• An automated and space efficient method to

accurately parse raw motion-data into a labeled sequence of surgical motions.

• Results support previous work that there exist quantitative differences in the varying skill levels of surgeons.

• Linear discriminant analysis is a useful tool for separating surgical motions.

22MICCAI 2005


Future Work

• Currently getting synchronized stereo video and API data. Will allow vision-based segmentation methods to complement our statistical methods.

• Apply to a larger set of expert surgeons to other representative surgical tasks.

• Create performance metrics to be used as benchmarks for surgical skill evaluation.

23MICCAI 2005


Acknowledgements

• Minimally Invasive Surgical Training Center at the Johns Hopkins Medical School (MISTC-JHU)

- Dr. Randy Brown, Sue Eller

• Intuitive Surgical Inc.- Chris Hasser, Rajesh Kumar

• National Science Foundation

Automatic Detection and Segmentation of Robot-Assisted Surgical Motions

Henry C. Lin, Dr. Izhak Shafran, Todd E. Murphy,

Dr. David D. Yuh, Dr. Allison M. Okamura,

Dr Gregory D. Hager

Thank you! Any questions?

Automatic Detection and Segmentation of Robot-Assisted Surgical Motions presented by Henry C. Lin...

Documents

Transcript of Automatic Detection and Segmentation of Robot-Assisted Surgical Motions presented by Henry C. Lin...