The AMIGO Project: An Advanced Multi-Modality Image-Guided Operating Suite

Surgical Planning LaboratoryBrigham and Women’s HospitalBoston, Massachusetts USA

a teaching affiliate ofHarvard Medical School

The AMIGO Project: An Advanced Multi-ModalityImage-Guided Operating Suite

Randy Ellis, Ph.D.

©2005 Surgical Planning Laboratory, ARR Slide 2

AMIGO: Phase II Layout


AMIGO: Central Operating Room


AMIGO: Phase I View to 3T MRI


AMIGO: Phase II View to PET/CT


AMIGO Applications, Phase I (2005-2006)

BRAIN: FUS (MRI, fMRI, DTI, T-MRI)

BRAIN: Open Surgery (MRI, fMRI, DTI,US)

BREAST: MRI-Guided Lumpectomy(MRI,3D-US,4D MRI of contrast

uptake)

BRAIN: Skull Base Surgery (MRI, fMRI, DTI, CT, US, Endoscope)


AMIGO Applications, Phase II (2006-2007)

BREAST CANCER: FUSLIVER THERMAL ABLATIONKIDNEY THERMAL ABLATIONCERVICAL CANCER BRACHYTHERAPYBONE METASTATSIS THERMAL ABLATIONBONE TUMOR FUSVERTEBROPLASTYPROSTATE BIOPSYPROSTATE BRACHYTHERAPYPROSTATE FUS


AMIGO Applications, Phase II (2007-2008)

SPINE SURGERY NEUROVASCULAR INTERVENTIONS CRANIOFACIAL SURGERYLIVER ENDSOCOPIC SURGERYKIDNEY FUS LUNG THERMAL ABLATIONLUNG BRONCHOSCOPYJOINT ENDOSCOPIC SURGERYCARDIAC EP ABLATION AVM EMBOLIZATION, AVM FUS


Image Intensive Environment

• Preoperative images:– diagnostic & planning

• Intraoperative images:– Updating anatomical changes

– Functional monitoring

• Postoperative images:– Verification


Imaging Modalities

Unite modalities in a patient coordinate frame:

• MR for soft tissues• PET for metabolic imaging• CT for bone surfaces• Fluoroscopy for updates• 3D ultrasound for soft tissues• Microscopes/endoscopes for sight• 1D signals (EEG, ECG, etc.)


Integrated Imaging and Intervention

+ Image/patient registration problem disappears

+ Real-time monitoring of procedures

- Visualization challenges

+ Multi-system assessment and treatment


Interoperability is Key

- Information management

• Maintain registration across rooms

• Multiple software systems

– GE/NAV for established procedures

– 3D Slicer, etc. for novel applications

• Multiple localization technologies

– Primarily EM & optical

– Robot also localizes


The AMIGO Navigator: An Overview of the Software Development Process

Randy Ellis, Ph.D.


Software is Expensive and Complex

• 31% were abandoned before

delivery

• Over 57% were a minimum of

189% over budget

Since then: overall, limited progress

Source: The Standish Group

An extensive 1994 study of thousands of US industrial projects found:


Our Guiding Principles

• Open Source:– Collaborative process– Free to commercialization

• Robust:– Use NAMIC/Kitware infrastructure– Automated testing of all changes

• Extensible:– Gain from ongoing Slicer activities– Expect to add new imaging modalities– Expect new interventional procedures


Application-Oriented Architecture

Base (common) software

MR

I Bio

psy a

pp

.

Neu

rosu

rgery

ap

p.

Bre

ast S

urg

ery

ap

p.


Storyboards for Applications

• Define the basic approach to the interventional procedure

• Prototype screen appearances (paper cartoons or PowerPoint are useful)

• Specify screen-to-screen actions

• Implement “happy-day” scenario

• Elaborate alternative workflows (e.g., error handling, new image acquisitions)

Result: fast development of well tested software

Engineer and clinicians work together:


Technical Overview

• Requirements drive the process• Application-specific:

– Rapid specification of ideal workflow– Catch design problems early– Elaborate necessary extensions only, not

all possible variants• Test-oriented:

– The specification produces tests before code is written

– Repeat all tests whenever changes are made

Result: robust, trustable systems


Example Application:

CT-Guided Brain Tumor Biopsy


Clinical Problem: Biopsy in CT Scanner

Idea:

• Perform brain biopsy using 2D slices

Technique:

• Attach fiducials to head holder

• Scan holder and patient

• Select slices showing structures at risk

• Superimpose needle trajectory on slices

• Verify biopsy with new CT acquisition(s)


CT Biopsy Phase 1: Procedure Workflow

1. Prepare patient (including fiducials)

2. Transfer patient to CT scanner

3. Acquire the CT scan (?inject?, image)

4. Load scan into Navigator

5. Software calculates registration

6. Software displays needle trajectory in CT

7. Perform the biopsy

8. Close the patient


CT Biopsy Phase 2: Software Actions

1. Load instrument descriptions

2. Load CT scan

3. Detect fiducials

4. Calculate registration

5. Select key CT slices

6. Track the needle guide

7. Overlay needle trajectory on slices


CT Biopsy Phase 3: Procedure Steps

1. Screen to select scan from disk

2. Screen to give registration progress

3. Screen to select CT slices

4. Screen to display navigation

Additional Requirements:

– Forward/backward buttons

– Exit button

– Tool status display (e.g., visibility)


LOAD SELECT NAVIGATEREGISTER

LOGO GOES HERE EXIT

Tool Status Display

CT Biopsy Storyboard: File Selection

All we need here is a simple dialog box



LOGO GOES HERE EXIT

Tool Status Display

CT Biopsy Storyboard : Registration

Need a progress bar during

registration, then report RMS error after registration



LOGO GOES HERE EXIT

Tool Status Display

CT Biopsy Storyboard: Selection Idea #1

Show original slices in here

SELECT SLICE 1/2/3/4

S

croll bar



LOGO GOES HERE EXIT

Tool Status Display

CT Biopsy Storyboard: Selection Idea #2

Show original slices in

here

Drag/Drop into here

S

croll bar



LOGO GOES HERE EXIT

Tool Status Display

CT Biopsy Storyboard: 2D Navigation

Slice #1

Slice #3

Slice #2

Slice #4

Show the needle trajectory on top of each slice


CT Biopsy Phase 5: Implement and Test

1. Create scans with/without fiducials (to test registration algorithms)

2. Build phantoms to measure navigation accuracy

3. Power cycle computer to test system robustness

• Unplug/plug tracking system, etc.


CT Biopsy Phase 6: Next Iteration

Next round, we may need to:

• Calibrate the biopsy needle intraoperatively

• Navigate the biopsy needle

• Use needle-oriented reslicing

• Migrate system to use MRI scans (new application: software re-use)

• Control MRI scanner for in-scanner biopsy (new application: software re-use)


Example Application:

Ventriculoscopy for Brain Tumor Biopsy


Clinical Problem: Deep Brain Biopsy

Idea:

• Use flexible scope to biopsy from within a brain ventricle

Technique:

• Same setup as for CT biopsy navigation

• Show 3D location of scope inside volumetric image

• Needs calibration of optical image to tracking device


Deep-Biopsy Phase 1: Procedure Workflow

1. Prepare patient (including fiducials)

2. Transfer patient to CT or MRI scanner

3. Acquire the 3D scan (inject & image)

4. Load scan into Navigator

5. Software calculates registration

6. Calibrate ventriculoscope

7. Software displays scope in volumetric scan

8. Perform the biopsy

9. Close the patient


Deep-Biopsy Phase 2: Software Actions

1. Load instrument descriptions

2. Load volumetric (MRI/PET/CET) scan

3. Detect fiducials

4. Calculate registration

5. Calibrate ventriculoscope to tracker device

6. Confirm calibration using a sharp probe

7. Track the ventriculoscope

8. Show scope in volumetric (MRI/PET/CET) scan


Deep-Biopsy Phase 3: Procedure Steps

1. Screen to select scan from disk

2. Screen to give registration progess

3. Screen to calibrate ventriculoscope

4. Screen to verify calibration

5. Screen to display navigation

Additional Requirements:

– Forward/backward/exit buttons

– “Tabs” to other tasks label “forward”


LOAD CALIBRATE VERIFY NAVIGATEREGISTER

LOGO GOES HERE EXIT

Tool Status Display

Deep-Biopsy Storyboard: File Selection

All we need here is a simple dialog box

ACCEPT



LOGO GOES HERE EXIT

Tool Status Display

Deep-Biopsy Storyboard: Registration

Need a progress bar during registration, then report RMS

error after registrationACCEPT



LOGO GOES HERE EXIT

Tool Status Display

Deep-Biopsy Storyboard: Calibrate Vscope

Show live video feed in here

CALIBRATE



LOGO GOES HERE EXIT

Tool Status Display

Deep-Biopsy Storyboard: Verification

3D probe model on top of live video feed here

ACCEPT



LOGO GOES HERE EXIT

Tool Status Display

Deep-Biopsy Storyboard: 3D Navigation

Live video feed from scope

Scope field-of-view cone inside

volume

Forward rendering from

scope


Deep-Biopsy Phase 5: Implement and Test

1. Create scans with/without fiducials (to test registration algorithms)

2. Calibrate optics to tracker device

3. Verify calibration with sharp probe

4. Build phantoms to measure navigation accuracy

5. Power cycle computer to test system robustness

• Unplug/plug tracking system, etc.


Deep-Biopsy Phase 6: Next Iteration

Next round, we may need to:

• Calibrate the biopsy needle

• Navigate the biopsy needle

• Use needle-oriented reslicing

• Acquire/integrate intraoperative MRI (new application: software re-use)

• Migrate to laparoscopic ultrasound (new application: software re-use)

The AMIGO Project: An Advanced Multi-Modality Image-Guided Operating Suite

Documents

Transcript of The AMIGO Project: An Advanced Multi-Modality Image-Guided Operating Suite