The Evolution of a VR-based IGT Environment

Terry Peters, Robarts Research Institute

David Gobbi, CEO, Atamai Inc.

IGT Workshop

MotivationMotivation

• Why AtamaiViewer?– IGT research requires software!– There was nothing out there in 1997– All IGT projects require similar components– Leverage VTK, MINC– Our own Tracking code

IGT Workshop

HistoryHistory

• 1997 – New lab, new possibilities– Evolution: MINC

• 1998-1999– Adopt VTK, add Yves Starreveld (Surgeon-Programmer)

• 2000 – company formed– Strong support from Robarts Institute

• 2004 – funding from CFI, SGI– VASST project begins– AtamaiViewer is christened

IGT Workshop

Software Development PhilosophySoftware Development Philosophy• Power to the students!

– Most students are not programmers– Python scripting to complement C++ core– Students should use each other’s software!

• Contribute to larger community– vtkImageReslice to vtkImageMapToColors (and

everything in between)– VTK Python wrapping– VTK on OSX

IGT Workshop

ArchitectureArchitecture

ITKITK

AtamaiVTKAtamaiVTK

Atamai Python ClassesAtamai Python Classes

AtamaiViewer “core”AtamaiViewer “core”

VTKVTK

Tkinter GUITkinter GUI

AtamaiViewer plugin modulesAtamaiViewer plugin modules

C++C++

PythonPython

IGT Workshop

ExpandabilityExpandability

AtamaiViewer APIAtamaiViewer API

“Core”“Core” VASSTVASST EpilepsyViewerEpilepsyViewer

Volume RenderingVolume Rendering

Data IOData IO

ElectrodesElectrodesMPRMPR

PackagesPackages

SkullStripperSkullStripper

fMRIfMRI

TrackingTracking

UltrasoundUltrasound

CineCine

ModulesModules

IGT Workshop

IGT Workshop

FeaturesFeatures

• Tracking– Polaris, Aurora, XarTraX, Flock of Birds

• Data Types– Images, meshes, point lists, transform files

• Visualization– Stereo (monitor, HMD or workbench)– Polygons, volume rendering, image slices

• Time-varying data– Cine cardiac viewing

IGT Workshop

FeaturesFeatures

• Multiple file formats– MINC, DICOM, VFF, Analyze, PNG, JPEG

• Expandability– Modules– Packages (collections of modules)

• customize front-end, change look and feel

IGT Workshop

ApplicationsApplications

• Epilepsy Viewer– Atamai product (3 sites)

• Stereotaxy Viewer– Frame-based (Parkinson’s)

• VASST– Augmented Reality

• AtamaiViewer “core”– Powerful in its own right

IGT Workshop

EP Atlas DatabaseEP Atlas Database

4

R50uA

PE112

111

0.220.040.2300

Starreveld et al, 2000

IGT Workshop

Deep brain surgery PlanningDeep brain surgery Planning

IGT Workshop

Guidance for Cardiac InterventionsGuidance for Cardiac Interventions

IGT Workshop

AR for port placement for Robotic CABGAR for port placement for Robotic CABG

IGT Workshop

Track the tools with magnetic tracking systemsTrack the tools with magnetic tracking systems

VR tools

US image

US Transducer

IGT Workshop

TEE image with virtual valve toolTEE image with virtual valve tool

IGT Workshop

..with clipping device and “pre-op image”..with clipping device and “pre-op image”

IGT Workshop

Delineating the dynamic targetDelineating the dynamic target

• Target identified solely from pre-op images is unreliable

• Interactively define target from US image

IGT Workshop

Clinical studyClinical study

IGT Workshop

Challenges facing engineering of IGT systems Challenges facing engineering of IGT systems • (non) availability of ubiquitous tracking

environments

• Non uniform software development environment

• Closed real-time systems (particularly US)

IGT Workshop

How can NCIGT help?How can NCIGT help?• Coordinate setting of standards for

collaborative software development.

• Provide management role

• Stimulate coordinated multi-institutional research in new tracking technologies

• Act as lobby group to pressure industry to make streaming 3D (esp US) more accessible from imaging systems