Mechatronics Lab

Workshop i Visualisering Presentation

Haptic and Visual Simulationof a Material Cutting Process

Using Patient Specific High Resolution CT-datafor Haptic- and Graphic rendering

Magnus G. Eriksson

Supervisor: Professor Jan WikanderCo-supervisor: Professor Hans von Holst

CTV (Center for Technology and Health Care)The Mechatronics Lab/Machine Design, KTH, StockholmDepartment of Neuronic Engineering KTH-STH, Huddinge

Mechatronics Lab

Background

Since 1980s

Since 1990

Since 1990s

Mechatronics Lab

Temporal Bone Surgery Simulator

Mechatronics Lab

Education of Surgeons

• “See one, do one, teach one”

• Patients risky situation

• Ethically and economically unacceptable

• Cadavers, plastic models or animals (high cost, ethical problems, difficulties of training results)

Exampel

Mechatronics Lab

Training in VR Simulators

• Avoid patients and cadavers

• Performance feedback

• Not time related

• Pre-operation planning

• Older, experienced surgeons

• Rare pathologies

• Reduce expensive costs

• First 60 patients <-> Simulator training (Ahlberg et al.)

Mechatronics Lab

VR and Haptic Simulators Used Today

Film example from Simulatorcentrum

• AccuTouch® Endoscopy Simulator

• LapSim® Laparoscopy Simulator

Metrics and Certified by US Food and Drug Administration (FDA)

Mechatronics Lab

Pos.

Pos.

ForceFeedback.

Pos.

”Real Force”

DrillingOperation

Anatomical Model

”Calc.Force”

VisualFeedback

VisualFeedback

surgeon

3D graphic

slave

master

The Temporal Bone Surgery System

patient

Mechatronics Lab

Research Goal and Focus

Goal:

• To develop a haptic and VR system for training and educating surgeons who practice bone milling

Focus:

1. To develop a VR system for realistic 3D representation of the human skull, including the changes resulting from the milling process

2. To develop an efficient algorithm for realistic haptic feedback to mimic the milling procedure using CT-data of the skull

Real time demands, without artifacts or delays when removal of material

Mechatronics Lab

System Design

SenseGraphics H3DAPI

Initialization

Graphic Thread 30 Hz

•3D Visualization•Material Removal

Haptic Thread 1000 Hz

•Collision Detection•Calculate Force

Mechatronics Lab

Possible VR Haptic and Milling Applications

• Temporal bone surgery

• Craniofacial surgery

• Dental tooth milling

• Vertebral operating procedures

• Freeform design

Research Goal: To develop a haptic and VR system for training and

educating surgeons who practice bone milling

Mechatronics Lab

Conclusion

• A haptic and VR surgical milling simulator prototype

• Patient specific DICOM data

• Efficient graphical rendering

• Haptic rendering to avoid fall-through problems

• Real time requirements when removing material

Film, tooth milling

Mechatronics Lab

Future Work

• Investigate various force models (3-DOF vs 6-DOF, mill is turned on-off, material removal rate) and benchmark

• Other applications, bone fractures, sculpting etc…

• User interface and virtual environments / visualisation

• Validate the simulator together with Simulatorcentrum

• Re-Design the haptik device and API (Matlab/Simulink)

• Investigate the economical and ethical benefits of using simulators

Film, skull bone milling

Mechatronics Lab

Initialization

Initialization

• Volumetric data from a DICOM-file

• Density and gradient 3D matrices

• Octree node structure containing the voxel data Range of x, y, and

z coordinates. Max/min density values.

Mechatronics Lab

Graphic Rendering

• Update rate > 30Hz and a latency of less than 300ms

Graphic thread, 30 Hz

Check for milling

Marching cubes algorithm on updated

tree nodes

OpenGL to create the shape of the object

Update density and gradient values

X_max

Y_max

r

d

Y_min

X_min

Film, skull bone milling

Mechatronics Lab

Haptic thread, 1000 Hz

Haptic Rendering

Collision detection

A force based on a proxy-probe method

Send force to the haptic device

If milling, add vibration force

proxyp

probep

1n̂

2n̂

2n

a1a

2a

tngproxyp _

)( _ probesurfaceproxy ppkF

Mechatronics Lab

Verification of the Haptic Algorithm

A Stiff Surface A Soft Surface