MMVR Workshop presentation1...Computers and Advanced Technology in Education, September 2008, Crete,...

H3D API An open source API for dexterous skills

simulators

Daniel Evestedt

SenseGraphics

OutlineWhat is haptics?◦ Concepts◦ Hardware devices◦ Software

H3D◦ HAPI◦ H3DAPI◦ Toolkits

Case studies◦ Simodont ‐Moog dental simulator

◦ Cork University hospital ‐ spinal anesthesia simulator

SenseGraphics technology

SenseGraphics H3DAPI Software development platform (Open Source)◦ SenseGraphics H3DAPI is designed for users intending to write software applications in “multi‐sensory”environments.

SenseGraphics Immersive Workstations◦ Work with your data in Immersive environment.

What is haptics?

Tactile vs kineastetic force feedback

Exploratory procedures

Lateral motion (texture)

Pressure (hardness)

Static contact (temperature)

Unsupported holding (weight)

Enclosure (global shape)

Contour following (local shape)

What is haptics?

3‐DOF ‐ 6‐DOF

Haptic devices Display systems

H3D platform

H3DAPI◦ X3D‐based scene‐graph API◦ Graphics, haptics, sound◦ Scripting through Python◦ C++, OpenGL

HAPI◦ Haptics SDK◦ C++

◦ Cross‐platform (Windows, Linux, OSX)◦ Device independent

www.h3d.orgw.h3d.o

HAPI

C++

Haptics API

Cross‐platform

Device independent

Modular and easily extendable

HAPI ‐ components

Force effects

Shapes andsurfaces Device handling

11

Haptics renderers

HAPI – program flow

Syncronisation point

Get device state

Render shapes and effects

Send force, torque to device

Shapes, effects

Contact info,Device info

Main thread

HAPI thread (1000Hz)

Code example (HAPI)int main(int argc, char* argv[]) {

AnyHapticsDevice hd;

HAPISurfaceObject *my_surface = new FrictionSurface();HapticPrimitive *my_haptic_sphere =

new HapticPrimitive( new Collision::Sphere( Vec3( 0, 0, 0 ),0.05 ),

my_surface );

hd.setHapticsRenderer( new GodObjectRenderer() );if( hd.initDevice() != HAPIHapticsDevice::SUCCESS ) {

cerr << hd.getLastErrorMsg() << endl;return 0;

}hd.enableDevice();hd.addShape( my_haptic_sphere );hd.transferObjects();

string temp_string;cerr << "Press ENTER to exit" << endl;getline( cin, temp_string );

hd.disableDevice();hd.releaseDevice();

}

H3D API

H3DAPI

H3DAPI Overview

X3D Python

HAPI

15

C++Scene Graph nodes

Field network

OpenGL

H3D API

X3D – scene definition

Python – scripting, scene updates, interaction

C++ ‐ nodes, scene building blocks

Simple X3D example (H3DAPI)

<Shape><Appearance><Material diffuseColor=“1 0 0”/><FrictionalSurface stiffness=“0.3” />

</Appearance><Sphere radius=“0.1” />

</Shape>

H3D API Components

3D graphics Shaders

Particle system

Haptics

AnimationScripting

Sound

More info

www.H3D.org

Wiki

Training

Consultancy

www.sensegraphics.org

Documentation

www.web3d.org

Toolkits

MedX3D (volume rendering)

VHTK (volume haptics toolkit)

UI (3D user interface)

RigidBodyPhysics(under development)

MedX3D

Web3D Consortium – Medical working group◦ Create ISO standard for representation of human anatomy

◦ First step: volume visualization

MedX3D toolkit for H3D API◦ First implementation of standard draft

◦ GPU based raycasting and slice based 3D texture rendering

Render styles

ComposedVolumeStyle

Compose several styles into one

Opacity-Phong-Edge enhancementOpacity-Phong

SegmentedVolumeData

Render different parts of data with different rendering styles.

Blood vessels: PhongHead: Opacity-Boundary

Cerebral ventricular system: ToneMappedHead: Opacity

VHTK – Volume Haptics Toolkit

Surface Haptics◦ interact with surfaces

Volume Haptics◦ haptic algorithms using volumetric data

◦ interaction with volumetric data

Applications◦ surgery simulation on authentic data

◦ scientific visualization of volumetric data

Dichloroethane Electro Potential

Femur Bone Drilling Dynamics in Volume Haptics

RigidBodyPhysicsRigid bodies◦ Physics properties (mass, friction, etc)

◦ Geometric shape

◦ Joints

◦ Haptic interaction

◦ Currently ODE and Nvidia PhysX

UI toolkit

3D/haptic UI nodes

Menus

Sliders

Buttons

Case study 1

Simodont ‐ dental simulator

32

1/8/2009

Moog

HapticMASTER

Conventional Dental Training Equipment

33

1/8/2009 34

SIMODONT

Visual Display

Dental Mirror

Haptic Burr

Courseware

Audio

Foot PedalFoot pedal

35

Visual DisplaySharp, true size collocated visual display

Full resolution, full stereo image projection and mirror technology

361/8/2009

Dental tools

Simulator and Courseware Interaction

37

1/8/2009

39

1/8/2009

Courseware

Case study 2

Spinal anesthesia simulator

Cork University Hospital and Interaction Design Centre, 2007

Spinal anesthesia – Can we improve how it is taught?


Determinant study – to understand the problem and the domainLearning related:

◦ Recognition of certain characteristic sensations as spinal anesthesia is performed

◦ Ability to visualise the relevant anatomy

Teaching related:◦ An explicit knowledge program for the procedure

◦ A case‐based learning program

◦ A valid, reliable competence assessment procedure

38

1/7/2009

Courseware


Modeling of sensations

Medical experts were recruited to rate sensations associated with spinal anaesthesia


Using the initial model

The haptic arm controls the needlesEach layer of tissue have different resistanceThe anatomy can be rotated for enhanced learning


Human tissue modeling

A human tisssue model was developed to reproduce the actual sensations of performing spinal anaesthesiaThe model has been verified with medical experts and is believed to be very accurate


Visualisations

Real‐time 2D and 3D visualisations are usedThe learner can place the needles and then view how the needle was placed and from that create a mental model of the procedure


Simulator‐supported assessment procedure

Thanks for listening!

See us in the exhibition hall for questions and demos

Referenceswww.h3d.org (developer community website, API downloads)

www.sensegraphics.com (the company behind H3D API)

www.web3d.org (X3D specification)

http://www.xj3d.org/extensions/volume.html (MedX3D Volume Rendering X3D component)

http://www.web3d.org/x3d/workgroups/medical/ (X3D Medical working group)

http://www.sensegraphics.com/index.php?option=com_content&task=view&id=24&Itemid=36(list of projects based on H3D API)

http://www.moog.com/simulationandtesting/ (Moog, the company behind the dental simulator)

http://www.medcap.eu/ (Spinal Anaesthesia simulator project page)

http://webstaff.itn.liu.se/~karlu/ (VHTK publications)

www.volvis.org (sample medical datasets)

http://dev.forsslundsystems.se (open source dental surgery simulator)

ReferencesCork Spinal Anaesthesia Simulator publicationsLövquist, E, Kulcsár, Z., Fernström, M., Aboulafia, A., Shorten. G., (2008) The Design of a Haptic Simulator for Teaching and Assessing Spinal Anaesthesia. In proceedings of the 5th Intuition Conference, 6‐8 October, Turin, Italy

Lövquist, E., Aboulafia, A., Breen, D., Shorten, G., Zhang, D., Albert, D., (2008) Designing a simulation‐supported adaptive assessment system for spinal anaesthesia. Proceedings of the 11th International Conference on Computers and Advanced Technology in Education, September 2008, Crete, Greece.

Exploratory proceduresSusan J. Lederman and Roberta L. Klatzky. Hand movements: A window into haptic object recognition. Cognitive Psychology, 19(3):343‐368, July 1987,

VHTK (Volume Haptics Toolkit)Karljohan Lundin Palmerius, "Direct Volume Haptics for Visualization", Ph.D. Thesis, May 2007

MMVR Workshop presentation1...Computers and Advanced Technology in Education, September 2008, Crete,...

Documents

Transcript of MMVR Workshop presentation1...Computers and Advanced Technology in Education, September 2008, Crete,...