Design Review

P15083: Virtual Visualization for Anatomy Teaching, Training and Surgery Simulation Applications

Agenda● Subsystem Design Overview

○ review of layer 1 - objective○ acquisition of BodyParts3D dataset○ UML - first iteration

● Detailed design - layer 1 breakdown○ model (3D mouse tutorial)○ control scheme○ selection method○ snap feature○ SME meeting - redesign

● detailed design - updated risks and tests○ risks○ tests○ future directions

Subsystem Design Overview

● only virtual components● An integrated stereo display● Surface models that are properly labeled● an interactive 3D interface via a 3d mouse.● creation of a preliminary anatomy visualization and learning module

Hardware integration is paramount during this layer, as it is essential for all other layers. Vizard was selected because of how easy hardware integration is reported to be.

Layer 1 prototype -

Skull Assembly prototype

● Implementation of a layer 1 prototype - skull assembly task○ skull pieces are distributed through the 3D

environment○ task is to assemble the pieces into the correct

relationship with each other○ student is provided with information about each

bone upon selection, correct bone positioning is rewarded with points, and a change in color.

3D bodyparts data set

● Free and open source (BSD)● includes bone, muscle, vasculature● segmented from MRI dataset and

retouched by artists at the center for life science in Tokyo. ● obj files are vizard compatible

Body Parts in 3DS Max

UML - first iteration● preliminary design of the overall structure of the game using UML formalism

Detailed Design - problem breakdown

● selection - how do we display what bone is selected, and toggle between manipulating the view and the bone?

● control scheme - what is the most intuitive control scheme, with the smallest learning curve possible?

● snap feature - when two pieces are brought in to coarse alignment, how to we make them snap together into a functional piece?

● data set, menu system to tie all together.

model for task - 3D mouse trainer

● spherical coordinate control system, allowing only rotation about the center point / origin ( intuitive navigation) 3DOF

● piece manipulation example, translation and rotation 6DOF

● selection scheme highlighting, however 2d selection w/ mouse

Control Scheme - mainview camera

● uses spherical coordinate, currently with a constant radius ( could be made variable if desired)

● Implementation originally attempted from scratch with displacement vectors, but was then implemented using vizard functions and object positional inheritance (lesson learned)

● key press events currenly manipulate the view position● vizact.whilekeydown(viz.KEY_DOWN,camcenter.

setEuler,[0,vizact.elapsed(-90),0],viz.REL_LOCAL)

Control scheme - bones● each bone should to rotate

about its own center of mass, rather than its origin location

● the centers of mass were calculated in meshlab, and stored in a CSV, then parsed in the script and used as centers of rotation

Selection Problem● wanted a 3D selection method rather

than a 2D mouse pointer● the position and orientation of the

glove is manipulated via the 3D mouse

● glove is imported into the scene as a 3D object, and upon selection (with space bar), a link is formed between the glove and the object to be manipulated.

Snap feature● two piece that fit together should

snap together when brought into close proximity and orientation (30 degrees)

● a bounding sphere sensor was added to the hierarchy of each of the bones

● the built in vizard proximity detection was used to detect when two pieces were adjacent

David Schwartz SME Meeting - the Game Loop

● event driven programming● encouraged to write functions in prototype, and then

refactor into class structure● encouraged to implement a single module first, instead

of fixating on creating the best general structure (analysis paralysis)

● discussion lead to:○ the push to develop menu system for skull game○ redesign of the UML chart

Menu system

UML chart, revised

Updated Risks

Tests - code● more difficult to write test for code that manipulate a 3D environment than

a script with an output metric● code testing will consist of an iterative process of

○ defining ideal behavior ○ speculating where breakdowns may occur○ designing code to create that ideal behavior, avoid breakdowns○ interacting within the environment to observe behavior, and identify

errors that were not predicted○ repeat

● ex) when manipulating the view, when one passes over the zenith (directly above) the view is inverted

Tests - human factorsNASA TLX Based Survey● Subjective workload tool to perform assessments on operators of devices.● Derives an overall workload score based on a weighted average across 6

different subscales:1. Mental Demands2. Physical Demands3. Temporal Demands4. Own Performance5. Effort6. Frustration Levels

Derive a survey based on these standards to give to students before next review.

Sample Survey

future directions● Most of the pieces of the demo are implemented either

in part or full now. ● The goals for the gate review are to:

○ finish implementing all desired features○ merge code into a coherent whole with classes, and

proper structure (functions => classes)○ test code

DemonstrationDemonstration of current capabilities