Algorithmic tools for real-time microsurgery simulation

Medical Image Analysis 6 (2002) 289–300www.elsevier.com/ locate/media

q†A lgorithmic tools for real-time microsurgery simulation

a , a a b*Joel Brown , Stephen Sorkin , Jean-Claude Latombe , Kevin Montgomery ,bMichael Stephanides

aComputer Science Department, Stanford University, Stanford, CA, USAbStanford–NASA National Biocomputation Center, Stanford, CA, USA

Abstract

Today, there is growing interest in computer surgical simulation to enhance surgeons’ training. This paper presents a simulation systembased on novel algorithms for animating instruments interacting with deformable tissue in real-time. The focus is on computing thedeformation of a tissue subject to external forces, and detecting collisions among deformable and rigid objects. To achieve real-timeperformance, the algorithms take advantage of several characteristics of surgical training: (1) visual realism is more important thanaccurate, patient-specific simulation; (2) most tissue deformations are local; (3) human-body tissues are well damped; and (4) surgicalinstruments have relatively slow motions. Each key algorithm is described in detail and quantitative performance-evaluation results aregiven. The specific application considered in this paper is microsurgery, in which the user repairs a virtual severed blood vessel usingforceps and a suture (micro-anastomosis). Microsurgery makes it possible to demonstrate several facets of the simulation algorithms,including the deformations of the blood vessel and the suture, and the collisions and interactions between the vessel, the forceps, and thesuture. Validation of the overall microsurgery system is based on subjective analysis of the simulation’s visual realism by different users. 2002 Elsevier Science B.V. All rights reserved.

Keywords: Microsurgery; Computer simulation; Surgical training

1 . Introduction instruments mounted on trackers. It generates a graphicrendering of the tissue deformations in real-time.

As computer power and graphics capabilities continue to Real-time simulation of deformable objects is needed inincrease, there is growing interest in surgical simulation as many areas of graphic animation, for example to generatea technique to enhance surgeons’ training. Such training cloth motions in animated movies or video games, tocurrently requires cadavers or laboratory animals. A com- provide realistic facial animation for digital actors, and toputer simulation option could reduce costs and allay ethical deform soft tissues in surgical simulations. Deformableconcerns, while possibly decreasing training time and objects raise a complex combination of issues rangingproviding better feedback to the trainees. However, for from estimating mechanical parameters, to solving largesurgical simulation to be useful it must be realistic with systems of differential equations, to detecting collisions, torespect to tissue deformation, tool interactions, visual modeling responses to collisions. See (House and Breen,rendering, and real-time response. This paper describes a 2000) for problems and techniques in cloth modeling andmicrosurgery training system based on novel computer (Delingette, 1998) for issues arising in surgical simulation.simulation techniques. The system allows a user to interact Many issues still lack adequate solutions, especially whenwith models of deformable tissues using real surgical simulation must be real-time.

Here we focus on fast and realistic simulation of tissueand suture, and detecting and processing collisions amongrigid and deformable objects. Our main goal is to develop

qElectronic Annexes available. See www.elsevier.com/ locate/media. efficient data structures and algorithms that can process*Corresponding author.

large models at a rate compatible with real-time graphicE-mail address: [email protected](J. Brown).† animation (30 Hz). To achieve this goal, we exploit theThis paper is based on earlier work appearing in (Brown et al.,

2001a,b). fact that many deformations arelocal. By propagating

1361-8415/02/$ – see front matter 2002 Elsevier Science B.V. All rights reserved.PI I : S1361-8415( 02 )00086-5

mailto:[email protected]

290 J. Brown et al. / Medical Image Analysis 6 (2002) 289–300

forces in a carefully ordered fashion through an elasticmass–spring mesh, we effectively limit the computationsto the portions of objects that undergo significant deforma-tions. To accelerate collision detection, we pre-computehierarchical representations for all objects in the scene;when objects are being deformed, we only update thoseparts of the hierarchies that need to be modified.

We have used these algorithms, along with other tech-niques, to build a system aimed toward microsurgicaltraining. Microsurgery is a well-established surgical fieldwhich involves the repair of approximately 1 mm vesselsand nerves under an operating microscope. It is a necessityin many reconstructive procedures, including the success-ful reattachment of severed digits. Using a forceps, thesurgeon maneuvers a suture (needle and thread) throughthe two ends of a severed vessel and ties several knots tostitch the two ends together. The two parts of the vessel

Fig. 1. Setup for microsurgery.undergo deformations caused by their interactions with thesuture and the forceps. The surgeon receives only visualfeedback, as the vessel is too small to produce any the following two sections. Tool simulation synchronizesperceptible reaction force. Microsurgeons typically acquire virtual surgical tools with real tools that are connected totheir initial skills through months of practice in an animal external tracking devices. Virtual tools consist of one orlab, at which point they still require months of supervision more rigid parts modeled as triangulated surfaces. Theirin the operating room. Without practice, these skills can positions and orientations are controlled by the externalquickly degrade. The need for such a suturing simulation devices at high update rates (typically 100 Hz), and otherhas been previously addressed in (Delingette, 1998), and a information from the devices may control relative rotationsperformance study in (O’Toole et al., 1999) discusses the or translations of the parts that make up one tool. Interac-validity of using such a simulator to develop surgical skill, tions between tools and deformable objects, such asalthough it does not provide technical details of the actual grabbing, poking and cutting, are dependent on the colli-simulation. Other aspects of suture simulation are dis- sion detection module described in Section 5.

¨cussed in (Kuhnapfel et al., 2000). The setup for microsurgery includes two real surgicalThe main contributions of this paper are the algorithmic forceps instrumented to detect closure and attached to

tools that we propose for simulating deformable tissue in electromagnetic trackers (miniBIRD of Ascension Tech-real-time. We provide extensive quantitative analysis of the nology Corporation). The user’s translation, rotation, open-performance of these tools. On the other hand, the ex- ing and closing of these forceps directly controls forcepsperimental microsurgery system has only been validated models in the simulation. Using the forceps, models ofthrough subjective visual analysis by several users. blood vessels can be grabbed and deformed. A suture can

Section 2 describes an overview of our simulation be manipulated to pierce through and realistically interactsystem. Sections 3, 4 and 5 present our simulation and with the vessels and the forceps. Stereo glasses allow thecollision-detection algorithms, with specific references to necessary depth perception to complete the task. Fig. 1the microsurgery simulator. Section 6 discusses current shows a user of the simulator.and future work. The system also supports parallel processing using

multithreading. Our implementation on a dual-processormachine (Sun Ultra 60, two 450 MHz processors) can use

2 . System overview two threads of execution to separate the simulation andcollision detection from the graphical rendering. In this

Our software system includes a deformable object way, visual updates occur at a guaranteed rate while thesimulator, a tool simulator, and a collision detection simulation continues uninterrupted.module. A graphics display allows multiple objects to berendered from a 3D virtual world onto the screen at 60 Hzor higher. The user has complete control of the view, and 3 . Computation of object deformationsmay use stereo glasses for true binocular depth perception(rendered at 30 Hz or higher per eye). The positions of the 3 .1. Relation to previous workobjects are read from the deformable object and toolsimulators before each screen refresh. Research on modeling deformable objects has increased

Deformable object simulation is described in detail in dramatically in the past few years. Most proposed 3D

J. Brown et al. / Medical Image Analysis 6 (2002) 289–300 291

models / techniques fall into two broad categories,mass– a surgery), where computations can be done off-line, butspring meshes and finite elements. must provide accurate, patient-specific results.

A mass–spring mesh is a set of point masses connectedby elastic links. It represents the tissue geometry and is 3 .2. Mass–spring elastic meshused to discretize the equations of motion. At each nodeNan equation defines the force exerted onN by the nodes to We represent the geometry of a deformable object by awhich N is connected. Mass–spring models have been 3D meshM of n nodesN (i51, . . . ,n) connected by linksiused in facial animation (Terzopoulos and Waters, 1990), L , i, j [ [1, n], i ± j. Each node maps to a specific pointijcloth motion (Baraff and Witkin, 1998), and surgical of the object, so that the displacements of the nodessimulation (Joukhadar and Laugier, 1997), to cite only a describe the deformation of the object. The nodes and linksfew works. They are relatively fast and easy to implement, on the object’s surface are triangulated, whereas the otherand allow realistic simulation for a wide range of objects, nodes and links are unrestricted, though it is often conveni-including viscoelastic tissues encountered in surgery. ent to arrange them in a tetrahedral lattice. Fig. 2 shows

Finite element methods (FEMs) use a mesh to de- the surface and underlying links of a mesh representing acompose the domain over which the differential equations severed blood vessel; this mesh contains 98 nodes and 581of motion are solved. The mesh represents the domainlinks. The more complex mesh in Fig. 3(a) consists ofinitially occupied by the object and the FEM technique 40,889 nodes and 212,206 links forming a tetrahedralcomputes a vector field representing the displacement of lattice.each point in this domain. For example, FEMs have been The mechanical properties (viscoelastic, in most surgicalused to model facial tissue and predict surgical outcomessimulation applications) of the object are described by data(Keeve et al., 1996a; Koch et al., 1996; Pieper et al., stored in the nodes and links ofM. A mass m and ai1995). They may be more accurate than mass–springdamping coefficientc are associated with each nodeN ,i imodels, but they are also more computationally intensive, and a stiffnessk is associated with each linkL . Theij ijespecially for complex geometries and large deformations. internal force between two nodesN and N is F 5i j ijSome systems use either mass–spring or FEM techniques2 k D u , whereD 5 l 2 rl is the current length of theij ij ij ij ij ij

¨depending on the situation (Kuhnapfel et al., 2000). Others link minus its resting length, andu is the unit vectorijuse preprocessing steps to reduce FEM computation (Bro-pointing from N toward N . The stiffnessk may bei j ijNielsen and Cotin, 1996; Cotin et al., 2000), and Picin- constant or a function ofD . In either case,F is aij ijbono et al. (2001) extends the ‘tensor–mass’ model of function of the coordinate vectorsx and x of N and N .i j i jCotin et al. (2000) to non-linear elasticity. This representation can describe objects that are nonlinear,

Other examples of mass–spring models, FEMs, and non-homogeneous, and anisotropic.alternate models are too numerous to cite here, but are At any timet, the motion/deformation ofM is describedcited in Delingette (1998) and Cotin et al. (1999). by a system ofn differential equations, each expressing the

Mass–spring meshes seem better suited for surgicalmotion of a nodeN ,itraining—the application domain considered in this

extm a 1 c v 1 O F (x , x )5m g 1F , (1)paper—which relies more on visual realism than exact, i i i i ij i j i ij[s (i )

patient-specific deformation, but requires that simulationsbe performed in real-time. In contrast, FEMs may address where x is the coordinate vector ofN , v and a are itsi i i i

better the needs of other applications (e.g., pre-operativevelocity and acceleration vectors, respectively,m g is theiextsurgical planning and predicting the long-term outcome of gravitational force, andF is the total external forcei

Fig. 2. Mesh example.


Fig. 3. Simulation of cutting operations (Courtesy C. Bruyns).

applied toN . s(i) denotes the set of indices of the nodes LetI be the set of indices of all the non-control nodes ofi

adjacent (connected by a link) toN in M. M, and letd be a constant time step (in our implementa-i

tion d is set to 1/30 s). At each timet 5 kd, k 5 1, 2, . . . ,the quasi-static simulator solves Eq. (2) for the positions of3 .3. Simulation algorithmall the non-control nodes. To achieve real-time animation,it returns these positions within timed. The algorithm isWe have developed a ‘dynamic’ and a ‘quasi-static’the following:simulator. The dynamic simulator uses classical numerical

integration techniques such as fourth order Runge–Kutta toAlgorithm QSS:solve Eq. (1). However, in many situations encountered in1. Acquire the positions of all the control nodessurgical simulation, a simpler algorithm based on quasi-2. Repeat until timed has elapsedstatic assumptions gives realistic results at a much faster

For everyi [ Irate. We describe this quasi-static simulator below.

(a) f ← O F 2m gi ij i3 .3.1. Assumptions j[s (i )

We refer to the nodes ofM that are subject to external(b) x ←x 1afi i iforces as thecontrol nodes. We assume that the position of

each such node is given at any time. In our surgicalStep 2 computes the residual force applied to each nodesimulation system, the control nodes correspond to theand displaces the node along this force. A conjugate-portions of tissue that are pulled or pushed by surgicalgradient-style method can also be used by moving the nodeinstruments or held fixed by bone structures or clampingalong a combination of the old and the new forces. Ideally,tools. The positions of the displaced control nodes arethe value of the scaling factora should be chosen as largeobtained online by reading the positions/orientations ofas possible such that the iteration converges. This choicetracking devices. We also assume that the velocity of thetypically requires experimental trials.control nodes is small enough so that the mesh achieves

The timeout condition of Step 2 guarantees that QSSstatic equilibrium at each instant. This is a reasonableoperates in real-time even as the size of the meshMassumption for soft objects with relatively high dampingincreases. Hence, Step 2 is not guaranteed to reach exactparameters, which is the case for most human-body tissues.equilibrium at every step, that is, someN ’s will have aiWhen these assumptions do not hold, the dynamicnon-zero forcef acting on them afterd amount of time. Asisimulator must be used.mesh size increases, each iteration of Step 2 will takelonger, and thus fewer loops will be possible in the3 .3.2. Quasi-static algorithmallowed time. By comparing the positions computed by

Under the above assumptions, we neglect dynamicQSS to the actual equilibrium positions (computed without

inertial and damping forces. The shape ofM is defined bytimeout), we can measure how the accuracy of the

a system of equations expressing that each non-controlsimulation degrades as the mesh complexity increases.

nodeN is in static equilibrium,i Step 2(b) updates the position of each non-control nodeusing the most recently computed positions of the adjacentO F (x , x )2m g 5 0. (2)ij i j i

j[s (i ) nodes, rather than those computed at the previous iteration


of Step 2. This scheme is most advantageous when the and error with subjective feedback about the quality ofnodes are processed in a wave-propagation order starting at deformations to setk 550 for the stiffnesses of springsthe displaced control nodes and expanding towards the within a layer andk 5100 for springs connecting adjacentnodes farthest away from any displaced node. This order- layers. This trial and error process was done under theing is computed by a breadth-first scan of the meshM: direct supervision of a microsurgeon. Fig. 2 shows a

smooth-shaded vessel and its underlying links. As the userAlgorithm NODE-ORDERING: displaces individual nodes of the vessels, the quasi-static1. Initialize I to the empty list algorithm described above is used to calculate the de-2. Mark the displaced control nodes inM to be at level 0 formation. Fig. 4 shows some examples of deforming the3. Fork 5 1, 2, . . . ,mark the unmarked nodes adjacent to vessels with forceps. Gravity is implemented as an external

a node at levelk 21 to be at levelk, and store them in force that can be either off or on, and in these figures it isI, until all non-control nodes have been marked. off, because the most interesting deformations are caused

by the user displacements.The displaced nodes may be arbitrarily distributed over

the mesh. The outcome of NODE-ORDERING is a listI of 3 .3.4. Performance evaluationnodes such that if indexi appears before indexj in I then The above algorithms are written in C11 and werethe level of N is less than or equal to that ofN . QSS tested on one 450 MHz processor of a Sun Ultra 60i j

processes the nodes as they have been ordered inI. workstation with 1 GB RAM. To address visual realism,Node ordering enables another major computational we asked surgeons to verify that the deformations were

savings. During an iteration of Step 2, if the positions of similar in shape and velocity to those encountered inall the nodes at some levelk are modified by less than a clinical operations. Figs. 4, 7 and 8 show such deforma-small pre-specified amount, then the algorithm stops tions.propagating the deformation further. In this way, the We did other experiments to quantitatively evaluate thenumber of levels treated at each iteration adjusts auto- performance of QSS. In particular, we created a regularmatically. This computationcutout is especially useful mesh whose nodes form a rectangular lattice. Each node iswhen object deformations are local. linked to its neighbors in thex, y andz directions. It is also

While QSS is being executed, any change in the set of diagonally linked to neighbors in thexy, xz andyz planes.displaced nodes only requires re-invoking NODE-ORDER- All links are given the same constant stiffness. Our meshesING to compute a new ordered setI. This set normally ranged from a 33333 box (27 nodes and 64 links) to achanges rarely (when nodes are grabbed or released), and 20320320 box (8000 nodes and 66,120 links).since NODE-ORDERING is a simple breadth-first scan of Given one such lattice mesh, we fix the nodes of anthe nodes, calling it adds a negligible computational cost to arbitrarily chosen ‘bottom’ face, and displace the middlethe simulation. Similarly, the mesh’s topology may change node of the top face upwards by one unit. We then runat any time. For example, in Fig. 3 cutting operations are QSS, but instead of running Step 2 for a fixed amount ofbeing performed. Links are removed from the mesh as they time, we do a fixed numberk of iterations (k5100, 50, 20,are crossed by the scalpel, so that the mesh used by QSS 10 or 5), where an iteration involves updatingf andx fori i

changes frequently. (In principle, such a cutting operation all the non-control nodes. We repeat several times thisviolates assumptions made above, and a dynamic simulator cycle of displacing the control node (by an additional unit)should be used. Nevertheless, QSS, which was used to and doingk iterations, and after each cycle we record theproduce Fig. 3, gives realistic results.) position of each node. Errors are computed as the distances

between these positions and the actual equilibrium posi-3 .3.3. Models tions. The equilibrium positions are found by running QSS

The vessels in our simulation are modeled as double- until the force on each non-control node is zero.hulled cylinders, with the inner and outer cylinders repre- Fig. 5(a) shows maximum and average errors for thesenting the thickness of the vessel. Each cylinder is different mesh sizes for 5 consecutive cycles of unitmodeled by several layers of nodes, with the layers evenly displacement followed byk 510 iterations. We see thatspaced, and each layer consisting of several nodes evenly errors increase slightly as the box grows from 27 to 216spaced around a circle. Each node is connected by nodes, but then minimally as size increases to 8,000 nodes.deformable links to its neighbors within a layer and in The larger boxes have many nodes which may be movingneighboring layers. There are also connections between the hardly at all, which contribute to lowering the averageinner and outer cylinders, which provide torsional stability, error. However, the maximum error follows the samepreventing the vessel from twisting excessively around its pattern at about twice the average error, and remains atlong axis. One end layer of each vessel is fixed in space, most about 10% of the magnitude of the displacement ofrepresenting the fact that the vessels are clamped down the control node for all mesh sizes.during surgery, and only a portion of their length can be Fig. 5(b) shows maximum error after one unit displace-manipulated. We set all massesm 5 1, and we used trial ment for different numbersk of iterations. It comparesi


Fig. 4. Forceps deforming severed vessel.

Fig. 7. (a) Suture pulling vessel down; (b) suture pulling two vessels together.

Fig. 8. (a) Suture wrapped around forceps; (b) suture colliding with self and vessel end.


Fig. 5. (a) Maximum (solid lines) and average (dashed lines) error versus box size, for 5 successive displacements of the control node. (b) Maximum errorwithout (solid lines) and with (dashed lines) computation cutout versus number of iterations.

joints. The joints allow two degrees of rotational freedom,QSS with and without the computation cutout describedwhile the links are rigid and short enough that the suturepreviously. The plots for two different boxes are shownshape appears smooth. By keeping the angles between thetogether, and we can see that errors are quite low in allfirst few links fixed, we can model a rigid needle at onecases, even for the largest box and the lowest number ofend of the suture.iterations. When using the cutout, errors do not strictly

To model the motion of the suture, constraint-baseddrop off to 0 as the number of iterations increases, but thetechniques are used. Any node of the suture may bemore important difference is the sharp decrease in the timeconstrained by another object in the system. For example,required per iteration. Table 1 displays the number ofone node might be grasped by a forceps, and thus itsiterations that QSS performs while maintaining a 30 Hzposition is constrained by the forceps. If the suture hasupdate rate, without and with cutout.pierced through a vessel, a node will be constrained by theThese experiments demonstrate that for the objectsposition of the vessel. Finally, if the suture is draped overgiven, we can reasonably maintain a small relative erroranother object, nodes will be constrained by that object.given only 10 or fewer iterations of QSS Step 2 per time

The motion is then calculated in a ‘follow-the-leader’interval. Furthermore, the cutout method can completemanner as follows: a constrained nodeN is moved by thethese iterations in the appropriate 1/30 second, and scale i

constraining object fromx to x . We then computeto meshes containing thousands of nodes with no signifi- i.old i.new

the new positionx of its neighborN as the point acant performance penalty. i11.new i11

distanced along the line fromx to x , whered isi.new i11.old

the (fixed) length of the link connectingN and N . Thei i11

same is done for nodeN . This motion is propagated up4 . Simulation of the suture i21

and down the suture toN , N , . . . andN , N , . . .i11 i12 i21 i22

until the next constrained node or one end of the suture isThe suture is deformable but not elastic, so the abovereached. Fig. 6 demonstrates this technique for four linksdeformation techniques based on mass–spring models areof a suture. For nodes between two constrained nodesNnot applicable. Instead, the suture behaves as a needle and i

and N , the preceding algorithm will compute two pre-thread, which can stretch minimally if at all, and has a j

liminary results, propagating fromN to N , and fromN tofree-form shape which is affected by gravity and direct i j j

N . These results are averaged to give the final position.contacts. To achieve realistic deformation, we model the i

Certain constraints are designated assoft, such as wheresuture as an articulated object: 200 short straight links arethe suture is piercing a vessel or draped over anothersequentially connected at nodes which act as spherical

Table 1Effect of computation cutout on simulation rate

Mesh Nodes Edges Iterations at 30 Hz Iterations at 30 Hzwithout cutout with cutout

63636 216 1440 24 2483838 512 3696 9.5 13

10310310 1000 7560 4.6 815315315 3375 27,090 1.2 720320320 8000 66,120 0.4 6


Fig. 6. (a) Constrained node 2 is moved; (b) nodes 1 and 3 follow, then 4, then 5.

object, whereas the forceps grabbing the suture is ahard and cutting one object with another (edge to face colli-constraint. The distinction is that the suture can slide over sions). In the remainder of this section, we presentand/or through a soft constraint, changing which node of background on collision detection and a new algorithm forthe suture is constrained. It may be the case that two collision detection among deformable objects.constraints move in opposing directions and cause thesuture to stretch between them. In that case, the suture will5 .1. Related workslide through a soft constraint to decrease the stretch, butwill break if between two hard constraints (in real surgery, Research on collision detection between rigid objectsit is not difficult to break the suture by pulling with both has a long history in robotics, graphics, and solid model-forceps in opposite directions). Additionally, if the suture ing. Two main families of methods have been proposed:is pierced through a vessel and is pulled on both ends, thefeature-based (e.g. Baraff, 1990; Lin and Canny, 1991;suture will pull the vessel, causing it to deform as in Fig. Mirtich, 1998) andhierarchical (e.g. Cohen et al., 1995;7(a). Fig. 7(b) shows the suture pulling together the two Gottschalk et al., 1996; Klosowski et al., 1998; Palmer andvessels. Grimsdale, 1995; Quinlan, 1994). A feature-based method

exploits temporal and spatial coherence in the geometricmodel to maintain the pair of closest features. A hierarchi-

5 . Collisions and interactions cal method pre-computes a hierarchy of bounding volumesfor every object. During a collision test, the hierarchies are

Interaction with virtual tools is a necessary component used to quickly discard large subsets of the object surfacesof any realistic simulator, and has attracted recent attention that are too far apart to possibly collide. Hierarchies using(Basdogan, 2001). Almost all object interactions depend at various primitive volumes have been proposed. Whilesome level on collision detection (Lin and Gottschalk, some volumes allow closer-fit approximation, they also1998). Grabbing is achieved by finding nodes colliding yield more costly intersection checks. Spheres give goodwith the tip of the grabbing object (e.g. forceps). Piercing results over a broad range of objects.the vessel requires finding a collision between the needle Although each approach has distinct advantages, theedges and a vessel face. Draping the suture around another hierarchical approach is better suited when objects areobject also involves edge to face collisions (Figs. 8(a) and highly concave. The main issue in using it with deformable8(b) show the suture around forceps and vessel), and objects is that pre-computed hierarchies may becomedraping it around itself requires edge to edge self-colli- invalid when objects deform, while re-computing new onessions. Other interactions modeled by the system (although at each collision query would be too time consuming.not specifically in the microsurgery simulation) include Below, we propose an algorithm that does not modify theprodding one object with another (face to face collisions), topology of the hierarchy representing an object, but only


updates the size and location of the primitive volumes Algorithm COLLISION (N , N )1 2

labeling the nodes of this hierarchy. Our algorithm derives 1 IfN and N have null intersection then return 01 2

from the one proposed by Quinlan (1994) for rigid objects. 2 ElseFewer works exist for deformable objects (e.g. Baraff (a) If bothN and N are leaf spheres then test the1 2

and Witkin, 1992; van den Bergen, 1997; Joukhadar and corresponding two triangles for collision; return 1 ifLaugier, 1997; Smith et al., 1995; Volino and Magnenat- they collide and 0 otherwiseThalmann, 1995). The algorithm in (van den Bergen, (b) IfN is smaller thanN then switchN and N2 1 1 2

1997) is the closest to ours. It also builds a tree of fixed (c) If COLLISION(N , left-child(N ))51 then return 11 2

topological structure. But this tree is not balanced (which Else if COLLISION(N , right-child(N ))51 then1 2

may seriously increase the cost of collision queries) and its return 1 Else return 0nodes are axis-aligned boxes. The main difference, how-

5 .3. Application to deformable objectsever, is in the tree-maintenance algorithm. Unlike van denBergen (1997), our algorithm exploits the locality of most

To use COLLISION, we must maintain the sphere treedeformations to minimize the number of node updates. Theof every deforming object. We propose a new sphere treealgorithm in (Volino and Magnenat-Thalmann, 1995) iswhose balanced structure is computed only once. When anspecifically aimed at detecting self-collisions of cloth-likeobject deforms, the structure of its tree remains fixed, i.e.,objects, an issue that we have not carefully studied so far.no sphere is ever added or removed; only the radii andpositions of some spheres are adjusted. Moreover, the

5 .2. Quinlan’ s algorithm maintenance algorithm performs adjustments only wherethey are needed.

5 .2.1. Sphere tree of an object5 .3.1. Construction of a sphere treeLet A be a (rigid) object represented by its triangulated

Let S be the triangulated surface of a deformable objectsurfaceS. Quinlan’s algorithm covers every triangle inSA in some initial shape. The pre-computed treeT for Awith small spheres of predefined radius´ and constructs andiffers from the one in (Quinlan, 1994) in two ways:approximately balanced binary treeT that has one leaf per1. Instead of tiling the triangles ofS with small equal-sphere of radius. Each other nodeN in T is a sphere that

sized spheres, we assign each triangle a single leafencloses all the leaf spheres of the sub-tree rooted atN. Tsphere ofT—the smallest sphere enclosing the triangle.is constructed by recursively partitioning the setE of leafHence, whenS undergoes a deformation, the number ofspheres contained in a sub-tree (initially the set of all leafleaf spheres ofT remains constant. Moreover, updatingspheres inT ) into two subsetsE and E of equal1 2the radius and position of the sphere enclosing acardinality, until each subset contains a single leaf sphere.deforming triangle is faster than computing a newThe partitioning operation tries to minimize the intersec-tiling.tion and the radii of the two spheres that respectively

2. The approximately balanced structure ofT is generatedenclose the leaf spheres inE and E . A technique to1 2in the same way, by recursively partitioning the leafpartition the setE first computes the box that is aligned tospheres into two subsets of equal size. But the radiusthe object’s coordinate frame and contains the centers ofand position of each non-leaf sphere is computed tothe leaf spheres inE. It then divides the leaf spheres alongenclose the sphere’s two children. This yields a slightlythe longest side of this box.bigger sphere than the one computed to contain thedescendant leaf spheres, but the computation is much

5 .2.2. Collision detection faster.Let T and T be the respective sphere trees of two1 2

(rigid) objectsA andA . A collision query is specified by 5 .3.2. Collision detection1 2

the position and orientation ofA relative toA . Collision COLLISION is used unchanged.1 2

detection is performed by a depth-first traversal ofT and1

T during which pairs of spheres from the two trees are 5 .3.3. Maintenance of a sphere tree2

examined. If two intermediate spheres have null intersec- Each deformation of one triangle ofS requires adjustingtion, then the leaf spheres they contain cannot possibly the radius and position of the corresponding leaf sphereintersect, and the traversal is pruned; otherwise the chil- and of all its ancestors up to the root ofT. Our algorithmdren of one of the two nodes are examined. If two leaf performs those changes only prior to processing a query.spheres intersect, the two triangles tiled by these spheres The operation is done bottom-up, using a priority queueQare explicitly tested for collision. ForN and N , the root of spheres sorted by decreasing depths inT. Q is initialized1 2

spheres ofT andT , respectively, the following recursive to contain all the leaf spheres that enclose triangles that1 2

algorithm either finds the colliding triangles and returns 1, have been deformed since the last update ofT. It is thenor returns 0 if there is no collision: used as follows:


Algorithm MAINTENANCE:While Q is not empty do

1. w←extract(Q)2. Adjust the radius and position ofw3. Insert(Q, parent(w))

The only spheres that are modified are those that containat least one deformed triangle. Each such sphere ismodified only once, even if it contains several deformedtriangles.

Clearly, MAINTENANCE will perform better whendeformations are local than when they are scatteredthroughoutS. More specifically, a local deformation ofSaffecting k,,s triangles, wheres is the total number oftriangles inS, results in a total update time of O(k 1 log s). Fig. 9. Dark curve: query time versus separating distance for non-

colliding objects; light curve: query time versus penetration distance forInstead, if thek triangles are spread over the leaves, thisobjects already in collision.cost can be O(k log s). However, in the worst case, if all

triangles have changed shape, the maintenance operationonly takes time O(s). tion. The query times for different relative positions of the

objects are shown in Fig. 9. When the objects are far apart,5 .4. Performance evaluation each query is extremely fast and takes on the order of

tenths of a millisecond. When they get closer than 1 unitThe above algorithms were implemented in C11. We together, query time grows quickly (dark curve) to an

give results of experimental performance tests on an Intel asymptote of just under 60 milliseconds, as more spheres400-MHz Pentium II processor, with 256-MB memory and in the trees had to be examined to rule out a possiblerunning Windows 2000. collision. Once the objects are in collision, the query time

drops sharply to under 10 milliseconds (light curve). This5 .4.1. Sphere tree construction sharp drop suggests that a timeout could be imposed on

The pre-computation of an object’s sphere tree need not COLLISION, with a relatively minor risk of not detectingbe particularly efficient, since it is done only once per a collision once in a while. We did similar experimentsobject, prior to any simulation. Our software runs in time after deforming the two objects and updating their sphereproportional to the number of triangles and takes on the trees using MAINTENANCE. We observed no significantorder of 0.1 milliseconds per triangle. degradation of the query times, even for rather large

deformations of the objects. The unintuitive explanation of5 .4.2. Sphere tree maintenance this result is that the topology of the sphere tree is largely

To evaluate MAINTENANCE we considered a surface shaped by the adjacency relation between triangles, whichS initialized to a flat horizontal 1003100 grid, with each itself remains invariant under any deformation of thesquare split into two triangles. Hence,S consists of 20,000 object.triangles. To create a local deformation ofS, we pick avertexV, a radiusr of deformation between 1 and 10, and 5 .4.4. Re-implementationa direction (upward or downward), all at random. Each After the above testing, the code was implemented asvertex within distancer of V is translated by a distance one module in our overall system. One change allows ainversely proportional to its distance toV. To create sphere to bound features which are not triangles, such asscattered deformations ofS, we repeat this process several the individual links of the suture. Another change allowstimes. After computing the sphere tree for the flat surface, the reporting of all pairs of colliding features, rather thanwe ran MAINTENANCE to update this tree after various just one.deformations. We measured a running time for MAINTE-NANCE of 0.06 milliseconds per deformed triangle.

6 . Conclusion and future work5 .4.3. Collision queries

We considered two objects. One is modeled as a flat We have designed new fast algorithms for simulating thesquare mesh of 8 by 8 units tessellated with 8,192 deformations of soft objects and detecting collisionstriangles. The other object is a spherical ball, 2 units in among deforming and rigid objects. These algorithms takediameter and tessellated with 1,024 triangles. We moved advantage of several characteristics of surgical training: (1)the ball along a straight path through the center of the visual realism is more important than accurate, patient-square mesh, from 64 units separation to one unit penetra- specific simulation; (2) most deformations are local; (3)


human-body tissues are well damped; and (4) surgical the National Aeronautics and Space Administration (NAS-instruments have relatively slow motions. Our simulator NCC2-1010), the National Science Foundation (IIS-exploits these characteristics to solve quasi-static equations 9907060), and the NIH National Libraries of Medicineusing a ‘wave-propagation’ technique that has an auto- (NLM-3506). It has also benefited from equipment giftsmatic computation cutout when deformations become made by Sun Microsystems and Intel. This paper was

´ ´insignificant. The collision algorithm exploits deformation improved by discussions with Cynthia Bruyns and Fredericlocality to minimize the number of updates in the hierar- Mazzella. Fig. 3 was generated by Cynthia Bruyns usingchical representations of the deforming objects. the deformable object simulator described in this paper and

These algorithms have been integrated into a virtual- the cutting algorithms described in (Bruyns and Senger,reality system for simulating the suturing of small blood 2001).vessels. This system has been used by plastic and recon-structive surgeons in our lab and at various exhibits, andsubjective feedback in the form of questionnaires distribut- R eferencesed at the year 2000 annual meeting of the AmericanSociety of Plastic and Reconstructive Surgeons deemed theBaraff, D., 1990. Curved surfaces and coherences for non-penetratingsimulation realistic and potentially very useful. One next rigid body simulation. Computer Graphics 24 (4), 19–28.

Baraff, D., Witkin, A., 1992. Dynamic simulation of non-penetratingstep would be experimental and clinical verification, byflexible bodies. Computer Graphics 26 (2), 303–308.having surgeons who are learning the procedure use this

Baraff, D., Witkin, A., 1998. Large steps in cloth simulation. In: ACMtool, and assessing the quality of their virtual repairs SIGGRAPH 98 Conference Proceedings, pp. 43–52.through measurements such as angle and position of vesselBasdogan, C., 2001. Simulation of instrument–tissue interactions andpiercing. We could then try to establish quantitatively how system integration. In: Medicine Meets Virtual Reality 2001,http: /

/eis.jpl.nasa.gov/|basdogan/Tutorials /MMVRTuto01.pdf..practicing with the simulator affects future quality of realBro-Nielsen, M., Cotin, S., 1996. Real-time volumetric deformablevessel repairs. Another step would be to quantitatively

models for surgery simulation using finite element and condensation.study the visual realism, by comparing the simulation with Computer Graphics Forum (Eurographics ’96) 15 (3), 57–66.video of real surgeries, using computer vision or other Brown, J., Montgomery, K., Latombe, J.-C., Stephanides, M., 2001a. Atechniques. microsurgery simulation system. In: Medical Image Computing and

Computer-Assisted Intervention – MICCAI 2001, pp. 137–144.We are also investigating other surgical applications.Brown, J., Sorkin, S., Bruyns, C., Latombe, J.-C., Montgomery, K.,While force feedback is irrelevant in microsurgery, it is

Stephanides, M., 2001b. Real-time simulation of deformable objects:critical in many other applications (Delingette, 1998). QSS Tools and application. In: Computer Animation 2001, pp. 228–236.can compute the force applied to each displaced control Bruyns, C.D., Senger, S., 2001. Interactive cutting of 3D surface meshes.node in an elastic mesh. But it does not achieve an update Computers & Graphics 25, 635–642.

Cohen, J.D., Lin, M.C., Manocha, D., Ponamgi, M.K., 1995. I-COLLIDE:rate compatible with haptic interaction (roughly 1000 Hz).An interaction and exact collision detection system for large-scaleTo connect our simulator to haptic devices, we areenvironments. In: Proceedings of ACM Interactive 3D Graphics

developing fast techniques to interpolate between forces Conference, pp. 189–196.computed by QSS (Mazzella, 2001). The elastic mesh Cotin, S., Delingette, H., Ayache, N., 1999. Real-time elastic deforma-model does not allow the explicit representation of incom- tions of soft tissues for surgery simulation. IEEE Transactions on

Visualization and Computer Graphics 5 (1), 62–73.pressibility constraints often encountered in human-bodyCotin, S., Delingette, H., Ayache, N., 2000. A hybrid elastic modeltissues. A technique proposed in (Keeve et al., 1996b) to

allowing real-time cutting, deformations and force-feedback forovercome this limitation is to apply artificial corrective surgery training and simulation. The Visual Computer 16 (8), 437–forces to surface nodes to keep the object’s volume 452.approximately constant. Extending our collision-detection Delingette, H., 1998. Towards realistic soft tissue modeling in medical

simulation. In: Proceedings of the IEEE: Special Issue on Surgerymodule to efficiently detect collisions of an object withSimulation, pp. 512–523.itself is another short-term goal.

Gottschalk, S., Lin, M.C., Manocha, D., 1996. OBB-tree: A hierarchicalThere are many other issues to consider, such as the structure for rapid interference detection. In: ACM SIGGRAPH 96

simulation of knot-tying (Larsson, 2001), the detection of Conference Proceedings, pp. 171–180.mesh degeneracies (e.g., when one link crosses another),House, D.H., Breen, D.E., 2000. Cloth Modeling and Animation. A.K.

Peters, Ltd.and the modeling of collision responses. As more issuesJoukhadar, A., Laugier, C., 1997. Dynamic simulation: model, basicare addressed in a simulation system, more algorithms will

algorithms, and optimization. In: Laumond, J.-P., Overmars, M. (Eds.),run concurrently, and their efficiency will become even Algorithms For Robotic Motion and Manipulation. A.K. Peters, Ltd,more critical. pp. 419–434.

Keeve, E., Girod, S., Pfeifle, P., Girod, B., 1996a. Anatomy-based facialtissue modeling using the finite element method. In: Proceedings ofIEEE Visualization, pp. 21–28.A cknowledgements

Keeve, E., Girod, S., Girod, B., 1996b. Craniofacial surgery simulation.In: Proceedings of the 4th International Conference on Visualization in

This research was conducted in the Stanford–NASA Biomedical Computing (VBC ’96), pp. 541–546.Biocomputation Center. It was supported by grants from Klosowski, J.T., Held, M., Mitchell, J.L., Sowizral, H., Zikan, K., 1998.

http://eis.jpl.nasa.gov/~basdogan/Tutorials/MMVRTuto01.pdf.












Efficient collisions detection using bounding volumes hierarchies of Palmer, I.J., Grimsdale, R.L., 1995. Collision detection for animationk-DOPs. IEEE Transactions On Visualization and Computer Graphics using spheretrees. Computer Graphics Forum 14 (2), 105–116.4 (1), 21–36. Picinbono, G., Delingette, H., Ayache, N., 2001. Non-linear and aniso-

¨Koch, R.M., Gross, M.H., Carls, F.R., von Buren, D.F., Fankhauser, G., tropic elastic soft tissue models for medical simulation. In: Proceed-Parish, Y.I.H., 1996. Simulating facial surgery using finite element ings of the IEEE International Conference on Robotics and Automa-models. In: ACM SIGGRAPH 96 Conference Proceedings, pp. 421– tion.428. Pieper, S.D., Laub, Jr. D.R., Rosen, J.M., 1995. A finite-element facial

¨Kuhnapfel, U.G., C¸ akmak, H.K., Maaß, H., 2000. Endoscopic surgery model for simulating plastic surgery. Plastic and Reconstructivetraining using virtual reality and deformable tissue simulation. Com- Surgery 96 (5), 1100–1105.puters & Graphics 24, 671–682. Quinlan, S., 1994. Efficient distance computation between non-convex

Larsson, A., 2001. Intracorporeal suturing and knot tying in surgical objects. In: Proceedings of the IEEE International Conference onsimulation. In: Medicine Meets Virtual Reality 2001, pp. 266–271. Robotics and Automation, pp. 3324–3329.

Lin, M.C., Canny, J.F., 1991. A fast algorithm for incremental distance Smith, A., Kitamura, Y., Takemura, H., Kishino, F., 1995. A simple andcalculation. In: Proceedings of the IEEE International Conference on efficient method for accurate collision detection among deformableRobotics and Automation, pp. 1008–1014. polyhedral objects in arbitrary motions. In: Proceedings of the IEEE

Lin, M.C., Gottschalk, S., 1998. Collision detection between geometric Virtual Reality Annual International Symposium, pp. 136–145.models: A survey. In: IMA Conference on Mathematics of Surfaces, Terzopoulos, D., Waters, K., 1990. Physically-based facial modelling,pp. 37–56. analysis, and animation. Journal of Visualization and Computer

Mazzella, F., 2001. The forcegrid: A buffer structure for haptic inter- Animation 1, 73–80.action with virtual elastic objects. Master’s thesis, Computer Science van den Bergen, G., 1997. Efficient collision detection of complexDepartment, Stanford University. deformable models using AABB trees. Journal of Graphics Tools 2

Mirtich, B., 1998. V-clip: Fast and robust polyhedral collision detection. (4), 1–13.ACM Transactions on Graphics 17 (3), 177–208. Volino, P., Magnenat-Thalmann, N., 1995. Collision and self-collision

O’Toole, R.V., Playter, R.R., Krummel, T.M., Blank, W.C., Cornelius, detection: efficient and robust solutions for highly deformable sur-N.H., Roberts, W.R., Bell, W.J., Raibert, M., 1999. Measuring and faces. In: Eurographics Workshop on Animation and Simulation, pp.developing suturing technique with a virtual reality surgical simulator. 55–65.Journal of the American College of Surgeons 189 (1), 114–127.

Algorithmic tools for real-time microsurgery simulation

Documents

Transcript of Algorithmic tools for real-time microsurgery simulation