Composing 6 DOF Tracking Systems for VR/AR

Bernd Schwald, Helmut Seibert, Michael SchnaiderComputer Graphics Center (ZGDV), Department Visual Computing

Fraunhoferstr.5, D-64283 Darmstadt, Germany{bschwald,hseibert,mschnaide}@zgdv.de

Abstract

This paper presents a concept for the composition oftracking systems in the context of Virtual and AugmentedReality (VR/AR) applications. Several independent 6 DOFtracking systems can be used simultaneously to accomplishthe interaction requirements for a given application. Thecontrol unit for tracking systems, processing measurementswith optional properties as time stamps and quality valuesis described as well as procedures for the coordinate sys-tem alignment of different tracking systems and the con-figuration of such a hybrid system. Considerations on theway accuracy errors might increase or decrease by com-bining tracking systems are included. The concept is moti-vated considering a medical AR system combining opticaland electromagnetic tracking technology as example.

1 Introduction

Tracking systems are an essential part of Augmented Re-ality (AR) systems and often used in Virtual Reality (VR)to allow more immersive human computer interaction. Oneof the basic decisions for the implementation of AR appli-cations is the choice of a convenient tracking system re-garding the environmental influences. Outdoor respectiveindoor scenarios, lighting conditions and necessary interac-tion ranges define the requirements for tracking systems.

Besides this, potential occlusion problems for opticaltracking systems or distortions caused by ferromagneticmaterials in the environment in case of electromagnetictracking need to be considered, too. Taking into accountall relevant conditions leads to the choice of one particu-lar tracking system in some cases, but in many other casesthe conclusion is, that no tracking system matches all re-quirements. As a consequence a hybrid tracking system,composed by different stand-alone tracking systems, mightbe the better choice to meet all constraints. These consid-erations are the starting point for the work presented in thispaper.

The variety of tracking systems is large, includingamong others different technologies like inertial, electro-magnetic and of course optical tracking systems. The con-cept, how to compose a hybrid tracking system out of stand-alone tracking systems is described in section 2. This con-cept is generic for the combination of tracking systems de-livering 6 degrees of freedom (6 DOF) in terms of rotationand translation of rigid bodies. While a combination with3 DOF tracking systems is feasible, this concept does notcover a combination with 2D video tracking systems.

In previous work Reitmayr and Schmalstieg [14] pre-sented an open software architecture called OpenTracker,allowing easier development and maintenance of hardwaresetups for VR/AR applications, based on XML. While theOpenTracker framework is more generic in terms of ex-change ability of tracking systems, the concept introducedin this paper puts more emphasis on the alignment betweencoordinate systems of sensors and tracking systems, seesection 2.3. The idea in our work is to provide all func-tionality needed for the implementation of an hybrid track-ing system consisting of several tracking systems in a singlemodule.

Baillot et al. [3] presented a tracker alignment frame-work for AR presenting an unified general calibrationmethod for calculating transformations relating the base ofthe tracking system to virtual world and the tracking sys-tem’s sensor to the graphics display. The multiple point cal-ibration technique proposed in their paper goes back to Parkand Martin in [13] and is planned to be used for hybrid sys-tems as well, but not evaluated for this purpose yet.

With the focus on mobile applications, Ribo et al. pre-sented in [15] a flexible software architecture for hybridtracking combining vision and inertial sensors, which is tar-geted to AR scenarios. In general, AR frameworks and thecombination of several tracking technologies are of growinginterest. ARToolKit [8] is a well known software library,which allows to implement marker-based AR applicationsrapidly. A generic design of a component-based AR frame-work was presented in [16]. The concept of the Dwarf sys-

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

tem shows in principle problems of flexible solutions in theAR context.

One of the most important properties of tracking systemsis the achievable accuracy. When combining tracking sys-tems, the accuracy errors of the stand-alone systems con-tribute to the error of the hybrid system, depending on theway they are combined. In our estimation, the combina-tion of tracking systems will not lead to a decreased errorof the 6 DOF measurements in most cases, but can limitthe disadvantages of stand-alone systems. In section 3 theinteraction of accuracy errors within a combined trackingsystem is considered.

The medical AR application described in section 4 isbased on the presented concept and combines an optical andan electromagnetic tracking system. It allows high accu-rate wireless interaction using an optically tracked transpar-ent display placed between physician and patient. Due toocclusion problems optical tracking below this transparentdisplay is problematic and therefore combined with electro-magnetic tracking of the medical instruments. For a betterunderstanding of the concept we will refer repeatedly to thisexemplary combination of tracking systems in this paper.

2 Concept for Composing Tracking Systems

Many VR/AR applications require more than one track-ing system to allow or improve all interaction possibilitiesneeded to satisfy the requirements of the user. The maintasks in composing tracking systems can be divided into twocategories:

• Determination of alignment transformations allowingto transform all tracking data into one common coor-dinate system and

• continuous transform of all measurements.

AR applications also require the registration of virtual in-formation to real world objects. The following sections in-troduce a concept for the implementation of a control unitwhich is intended to provide the desired functionality forarbitrary VR/AR applications.

2.1 Data Flow and Configuration

Each 6 DOF tracking system addressed here determinesposition and orientation of special sensor devices. Eachtracking device is autonomously performing measurementsat a measurement rate defined by the system and more orless affected by environmental influences. Optical track-ing systems can be heavily disturbed by occlusions, electro-magnetic devices become imprecise by increasing the dis-tance between sensor and emitter. Some systems report a

quality value for the measurements, which allows the eval-uation of measurements in a following process. To reflectthese properties of tracking systems we decided to integrateeach tracking system as independent module emitting avail-able measurement data sets including all information likequality value and time stamp information via network to asingle control unit.

.

.

.

.

Application

controller setup

registration procedures

user interaction options

management of virtual objects

(interaction devices) (other devices)(display, viewpoint)

OpticalTracking Tracking

Electrom. State Sensor

Input Device Management

Output Management

Co

ntr

ol U

nit

RenderingModule

AdditionalFeedback

algorithmsAlignment/registrationTransformations

Figure 1. Application data flow. All data isprocessed by the control unit.

The control unit, see figure 1, is designed to apply allnecessary transformations in an appropriate way and to pro-vide the transformed data for the main application. All com-ponents like tracking systems and their sensors are mappedto a hierarchy of software components which represent thehardware components and hold all relevant information.

The flexible concept requires a configuration, which hasto be defined once for each application and allows to changethe setup easily. The configuration includes informationabout each tracking device, the number of sensors attachedto it, the corresponding alignment transformation and aunique identifier for network communication.

A sensor specific configuration keeps the sensor iden-tifier for network communication, corresponding trackingsystem, and the offset transformation which has to be ap-plied in its local coordinate system. The incoming measure-ments are stored in an event queue, which allows to applyfiltering on the incoming data. The parameters for eventprocessing and filtering need also to be defined.

Besides filtering measurements of each tracking sensorseparately, it is possible to define hybrid sensors, whichcombine the measurement data of different tracking sensorsby means of sensor fusion. Considerations concerning sen-sor fusion are discussed in section 3. After the creation of aconfiguration, it can be used to save and restore the state ofthe application.

All functionality concerning the alignment of different

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

tracking systems and registration issues is also provided bythe control unit. The alignment algorithm records and eval-uates a set of points with their representations in the coordi-nate systems of both trackers to determine the correspond-ing alignment transformation (see section 2.3). The execu-tion of configurable alignment and registration procedureshas to be triggered by the main application during the workflow.

Once initialized, the control unit is continuously col-lecting the measurements sent from all tracking systems.All necessary transformations in order to provide the mea-surements in the common coordinate system are applied toeach incoming measurement. The transformed measure-ments are emitted via network to supply the rendering mod-ule with the actual poses. Additional information availablefrom other interaction devices is also sent to the main appli-cation.

2.2 Coordinate Transformations

The overall transformation for each measurement of atracking system consists of four steps. This transformation,see equation 5, converts a point Xl given in a local coor-dinate system into his representation Xw in the common(world) coordinate system. While two of the four transfor-mations are fixed after calculating them in calibration pro-cesses, the two other transformations are changing, as theyare the output of the tracking system. As shown below inequation 6 the fixed and the dynamic transformations arealternating, preventing the simplification of the transforma-tion by merging the fixed parts. Usually it is not feasible

TRsens0

lTR

sens1

lTR

sens0

lTR

sens1

l

TRtr1

sens1TR

tr0

sens1TR

tr0

sens0

TRal

tr0

TRw

al

TRtr1

sens0

TRal

tr1

sensor 1sensor 0 sensor 0

Tracker 1

sensor 1

Tracker 0

Figure 2. Transformation hierarchy, the direc-tion of the arrows shows how the consecutivetransformations are applied.

to place the tracking sensor in a way that the desired po-sition and orientation can be obtained directly. Thereforea local rotation Rsens

land translation (offset) T sens

lis re-

quired to transform the tracker output Rtrsens and T tr

sens. Theindices of these transformations have the following mean-ing: Rtr

sensdesignates the rotation from sensor to tracker

coordinate system and T tr

sens the translation of the sensor intracker coordinate system. Analogous for the other trans-formations.

In case of a tracked interaction device, T sens

lis typically

the offset from the tip of the interaction device to the sensor,in case of a display device, this is the transformation fromthe sensor on the display device to the origin of the screenand in case of the user, it is the transformation from a sensorat the head to the real eye position. Equation

Xsens = Rsens

l · Xl + T sens

l (1)

allows a transformation of a certain point Xl , given in a lo-cal coordinate system to the coordinate system of the sensor.Translation T sens

land rotation Rsens

lhave to be determined

in a calibration process as described in 2.3 and are fixed. Inthe second step the output from the tracking system Rtr

sens

and T tr

sensis applied to Xsens resulting in Xtr analogous to

equation 1 by

Xtr = Rtr

sens · Xsens + T tr

sens. (2)

In order to combine the output of several arbitrary track-ing systems the reference coordinate system of each track-ing system has to be aligned to a common coordinate sys-tem in the next step. Therefore the measurements of eachtracking system have to be transformed with an additionalalignment transformation. Similarly to the last steps, thetransformations Ral

tr and T altr convert Xtr into its presenta-

tion Xal in a common coordinate system by

Xal = Ral

tr ·Xtr + T al

tr . (3)

Raltr and T al

tr are typically the fixed transformation from onetracking system to another (that means one of them is al-ready the shared coordinate system).

As a final step all coordinates are transformed to an arbi-trary coordinate system, for example if the application sce-nario includes a central object which defines the commoncoordinate system, and all measurements are required rela-tive to it. This final transformation consists of Rw

aland Tw

al

and is applied to Xal by

Xw = Rw

al·Xal + Twal (4)

with Xw as final result given in the common (world) coor-dinate system. Adapting the transformations (equations 1,2, 3 and 4) consecutively yields

Xw = R · Xl + T, (5)

with

R = Rw

l = Rw

al︸︷︷︸

dynamic

· Ral

tr︸︷︷︸

fixed

· Rtr

sens︸ ︷︷ ︸

dynamic

·Rsens

l︸ ︷︷ ︸

fixed

(6)

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

for rotation and

T = Tw

l

= Rw

al · (Ral

tr · (Rtr

sens · Tsens

l + T tr

sens)

+T al

tr) + Tw

al(7)

for translation. R and T transform a point Xl, e.g. thetip of a interaction device given in local coordinates, to itsrepresentation in world coordinates Xw. This transforma-tion fulfills all needs for every object to be tracked and tobe known in one arbitrary but common coordinate system.The hierarchy of all transformations is shown in figure 2.

2.3 Alignment and Registration

The usage of several tracking systems in an AR/VR ap-plication presumes that all measured positions and orienta-tions are available in one common coordinate system. Usu-ally each tracking system defines its own coordinate sys-tem. If more than one tracking system is used, the output ofthe different systems has to be aligned, to appear in a com-mon coordinate system. Depending on the application andthe particular combination of tracking systems, customizedalignment respectively registration processes have to be per-formed.

The alignment routine as part of the control unit pre-sumes the use of a special alignment device, see figure 3,with sensors of each tracking device attached. Once trig-gered, the procedure starts to collect all incoming track-ing data until sufficient measurements are available. Cor-responding measurements with respect to the time stampinformation are used to determine the corresponding align-ment transformation. During an approximately 20 sec-onds measurement period the registration device has to beswiveled inside the interaction area of both tracking sys-tems. The result is applied to all following measurements ofthe appropriate tracking system. While each tracking sys-tem can have several sensors, the alignment procedure hasto be performed only once for each combination of trackingsystems, independent of the number of sensors.

The alignment can be reduced to the problem of calcu-lating a transformation consisting of rotation R and transla-tion T from a set {Xi, Xi}i=1..n of corresponding 3d pointsX = (x, y, z). Algorithmic solutions for this problemhave been discussed, evaluated and compared with other ap-proaches in [18].

To verify the result, the points Xi are transformed withthe calculated rotation R and translation T . The trans-formed points are compared with the points X i by statis-tical means. This allows an evaluation of the quality of theresult and if necessary the rejection of the result.

Further registration procedures are needed to register vir-tual objects to a real scene, the most fundamental task in AR

Figure 3. Alignment device with attached opti-cal marker and encased electromagnetic sen-sor.

scenarios. Generally correspondences between real and vir-tual objects have to be defined and the corresponding align-ment needs to be calculated. Real landmarks with knowncorrespondences in the virtual object have to be navigatedwith a tracked registration device to obtain the coordinatesin the common coordinate system. The alignment algorithmmentioned above can then be used to obtain the alignmenttransformation for the virtual object.

3 Accuracy error induced by Composition

As mentioned in the introduction, the accuracy errors ofthe stand-alone systems contribute to the overall error of thehybrid system, depending on the way, how they are com-bined. In fact, we do not expect to get an increased accuracyfor a combined tracking system compared to the highest ac-curacy of a single stand-alone tracking system.

An example how errors of tracking systems and align-ment or registration procedures can interfere is given in fig-ure 4. This example shows a combination of an optical andan electromagnetic tracking system for a quite typical ARapplication and is a generalization of the application pre-sented in section 4. The objects to track are

• a display device like an AR Window [10] or an HeadMounted Display (HMD),

• the head respectively eye position of the user,

• a subject or object, which is manipulated,

• an instrument or device for interaction with the sub-ject/object.

If an HMD is used, the first item in the list above is rathera registration problem between HMD an the second item,the eye position of the user, than a task of tracking. This

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

Tracking

Accuracy of

Augmentationof interaction

AlignmentAccuracy of

TrackingElectromagnetic

Accuracy ofobject

registration

of objectAugmentation

Optical Tracking Alignment(Algorithm)System

ElectromagneticTracking System

device

NavigationAccuracy

Object registration(Algorithm)

Accuracy ofOptical

Figure 4. Error scheme of a combined track-ing system. Arrows mean ’depends on’.

registration, better known as calibration of intrinsic param-eters of an HMD, is still a challenging task in AR technol-ogy and discussed in many papers [1, 21, 3] and omitted inthis paper.

While the first two objects in the above list are trackedusing an optical tracking system, to allow free movementfor the user, the two remaining objects are tracked by anelectromagnetic tracking system to avoid occlusion prob-lems. The pointing direction of the arrows in figure 4 repre-sent the relation ’depends on’.

Obviously, the accuracy of the alignment of two track-ing systems depends on the accuracies of the two trackingsystems and the alignment algorithm itself. In this setup allaugmentations depend on the accuracy of the tracker align-ment, because interaction devices and objects are trackedby an electromagnetic system, while user and display tech-nology are tracked using an optical system. In contrast tothe precision of overlays/augmentations, the navigation ac-curacy, that means the accuracy of the localization of theinteraction device relative to the object, is independent ofthe alignment of tracking systems, because only one track-ing system is used for both, the tracking of the interactiondevice and the object to be manipulated.

Figure 5 illustrates the transformation from interactiondevice to object based on figure 2 from section 2.2, definingTracker 0 as the electromagnetic tracker, sensor 0 trackingthe interaction device and sensor 1 tracking the object. Thecircular line represents the transformation

(TRsens1l )−1 · (TRtr0

sens1)−1 · TRtr0

sens0 · TRsens0l ,

which is independent of TRal

tr0.Applying this scheme to the medical AR system in sec-

tion 4 means that the precision of augmentations of instru-

device objectTR

sens0

lTR

sens1

l

TRtr0

sens1TR

tr0

sens0

TRal

tr0

TRw

al

Tr

sensor 0 sensor 1

Tracker 0

Figure 5. The circular line is the transforma-tion from interaction device to object. Thistransformation is independent of the trackeralignment.

ments and patient decreases by the error induced by thealignment of the tracking systems in comparison to a setupwith only one tracking system. However the accuracy of thepure navigation with the medical instrument inside the pa-tient is more important than the accuracy of augmentations.Note that the quality of the navigation is not influenced bythe alignment of tracking systems.

Another possibility to combine tracking systems is not touse them in parallel, as described above, but to fuse sensors.Figure 6 shows the same setup as discussed before, but withfused sensors for object registration/tracking and interactiondevice tracking. In this case, the advantages of a fused sen-sor are the possibility to reduce the distortion problem ofelectromagnetic sensors in metallic environments and oc-clusion problems for the optical tracking system.

Several ways of sensor fusion are possible. A straightforward approach is fusion by averaging available transfor-mations, additionally the fusion can be done with a pref-erence on the measurements of one of the sensors possiblyusing Kalman filter [22]. The advantages of sensor fusionstrongly depend on the method used for the implementa-tion. However in all cases, the navigation accuracy is nowdependent of the alignment of tracking systems and there-fore influenced by the alignment error.

Many other schemes for the combination of tracking sys-tems can be designed, especially, if the number of trackingsystems and or the number of their sensors increases. How-ever, one of the important considerations will be, whethertracking systems are combined in the sense of a parallel us-age or with real sensor fusion.

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

Tracking

of objectAugmentationNavigation

Accuracy

Accuracy ofobject

registration

Augmentationof interaction

AlignmentAccuracy of

TrackingElectromagnetic

Accuracy of

Optical Tracking Alignment(Algorithm)System

ElectromagneticTracking System

device

Object registration(Algorithm)

Accuracy ofOptical

Figure 6. Error scheme of a combined track-ing system with fused sensors for registra-tion and. Arrows mean ’depends on’.

4 Application in a medical AR system

As already mentioned in the previous sections, the im-plementation of the presented concept was performed as amedical AR system for supporting minimal invasive inter-ventions.

Minimal invasive interventions require the navigation ofinstruments inside the patient’s body without a direct line ofsight. On the other hand, medical imaging modalities likeComputer tomography (CT) or Magnetic Resonance Imag-ing (MRI) provide information about the anatomic struc-tures beyond the surface. Both, the augmentation of thephysician’s view with this information and to provide nav-igation support is a common task in medical VR/AR appli-cations. The medical domain offers a lot of work alreadydone in this area.

Approaches how to provide an ”x-ray view” to the physi-cian in the past, can be found among others as part of an ARoverview [2]. Early applications can be found in [11, 9, 7].While some related medical applications date back until fif-teen years ago [23], new AR navigation prototypes for com-puter assisted surgery were developed in the recent years.The tracking systems used for these systems cover electro-magnetic trackers as well as optical systems [19, 20, 4].

One of the basic decisions for the implementation of anAR application is in addition to the choice of tracking tech-nology, the kind of device which should be used to dis-play the augmentations. Due to the restricted options videosee-through Head Mounted Displays (HMD) or the harderto handle optical see-through HMDs are applied in manycases. Especially in the medical domain the restricted field

of view and the reduced quality of the view caused by theusage of a HMD are unpleasant limitations.

Figure 7. Pivoting transparent display usedfor augmentations. Active infrared markersare attached to its housing. Below the dis-play an anthropomorphic phantom used fordemonstration purposes.

Alternative display devices for AR are very rare, es-pecially under the demand to use a see-through device[5, 6, 12]. A transparent display, from now on called AR-Window, see figure 7, has been developed in the context ofthe MEDARPA (Medical Augmented Reality for Patients)[10] project. The usage of the AR-Window had impact onthe choice of tracking systems, as discussed later.

One typical medical scenario focused by our AR appli-cation is to support the interstitial brachytherapy, which al-lows to irradiate a malignant tumor ”from the inside”. Ir-radiation probes (hollow needles made of metal or flexibleplastic catheters) need to be implanted percutaneously intothe tumor. After a CT-based three-dimensional planning ofthe brachytherapy, the irradiation is applied with a smallradioactive source, inserted into the irradiation probe by acomputer-controlled after loading device and thus into thetumor itself.

The support for this application consists in a projectionof the three dimensional CT-image to appear visible at thereal location. Preoperatively defined target regions insidethe tumor are visualized as well as the virtual model ofthe irradiation probe mentioned above. The application de-mands the tracking of the user and one or more real worldobjects. The concept, how tracking systems are combinedfollows the generic setup sketched in section 3 in figure 4:

• The physician as user of the system, whose viewpointhas to be determined,

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

• the AR-Window as device for displaying the augmen-tations, see figure 8,

• the patient as real world object, equipped with land-marks placed on his body before image acquisition(CT, ...),

• a medical instrument as further real world object fornavigation inside patient’s body

• and an interaction device used for registration pur-poses.

The virtual information registered properly to the realworld has to be supplied in real time to be presented ona suitable media, in our case the AR Window, to augmentthe real scene.

The display and the user’s head are equipped with ac-tive infrared markers to allow optical tracking. This taskis performed by the EOS system [17]. The instrument aswell as the registration device are equipped with sensors ofan electromagnetic tracking system (Ascension pciBIRD).The properties of the tracking systems can be found in [18].

The combination of an optical with an electromagnetictracking system was preferred, because the instrument ismostly used below the display device which causes occlu-sion problems inhibiting optical tracking.

As preparation of the AR System, the tracking systemshave to be aligned, what simply consists in the followingprocedure: The user has to sway around an alignment de-vice, consisting of a plastic handle and an infrared diode,see figure 3. This has to be done for about 20 seconds, untilaround 250 measurements are recorded, from each system.The user has to take care, that he is in the interaction rangeof both tracking systems. After recording measurements,few more seconds are needed for the optimization until thewhole procedure is finished.

To prepare the registration of the patient, he has to beequipped with at least four, better up to ten spherical mark-ers. A volume image of the patient’s body and the markersis acquired by a CT- scan, the markers are identified in thevolume data and their 3D position is stored. The markersremain on the patient until the registration process with theAR system is finished.

Right before the usage of the AR system, the patientis registered with a special device. This device consistsin a handle with an integrated electromagnetic sensor anda short concave tip that fits to the surface of the spheri-cal markers. This finishes the preparation of the AR Sys-tem and it is ready to use. A more detailed descriptionof these processes including algorithms and the evaluationcan be found in [18]. The standard deviation of the posi-tional alignment results have a standard deviation of lessthan 2mm. For the registration of the patient, this value is

around 1mm. As described in section 3 the alignment er-ror does not influence the navigation error in contrast to thepatient registration error.

Figure 8. Transparent display augmentingreal view with additional information.

The provided navigation assistance consists now in vir-tual objects based on pre-processed medical image data anda representation for the irradiation probe used for naviga-tion, which are augmenting the scene. The navigation itselfis supported by changing colors of the instruments overlay,depending on the orientation of the instrument and its dis-tance to target regions.

5 Conclusion and Future Work

In this paper we presented a concept for the hybrid useof several 6 DOF tracking systems. The data flow, transfor-mations necessary to provide the measurements of severalsubsystems in a common coordinate system and alignmentand registration procedures were discussed. How errors candepend on the design of the combination of tracking sys-tems was described for a special setup.

Our concept for the hybrid use of several 6DOF trackingsystems is designed for easy configuration and exchange-ability of tracking systems. This is achieved on the one handby a modular software design and on the other hand by pro-viding the necessary registration/alignment procedures.

The concept allows to use tracking systems for differenttasks as outlined in the medical application. Furthermorethe fusion of sensors to perform the same task by two track-ing systems in order to reduce disadvantages of a singletracking system is supported by the design of the software.The algorithmic part for a sensor fusion is not discussedin this paper, but will be examined further, taking various

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE

existing approaches using Kalman Filter or other filter andprediction methods into account.

This concept is implemented and was motivated by amedical AR system [10], which has the intention to alloweasy integration of new tracking technology for navigationin the context of minimal-invasive surgery.

References

[1] R. Azuma and G. Bishop. Improving Static and DynamicRegistration in an Optical See-through HMD. In Proceed-ings of SIGGRAPH94, Computer Graphics, Annual Confer-ence Series, pages 197–204, 1994.

[2] R. T. Azuma. A Survey of Augmented Reality. Teleoper-ators and Virtual Environments 6, pages 355–385, August1997.

[3] Y. Baillot, S. Julier, D. Brown, and M. A. Livingston. ATracker Alignment Framework for Augmented Reality. InACM International Symposium on Mixed and AugmentedReality (ISMAR 2003), pages 142–150, Tokyo, Japan, Oc-tober 2003.

[4] W. Birkfellner, K. Huber, F. Watzinger, M. Figl, F. Wan-schitz, R. Hanel, D. Rafolt, R. Ewers, and H. Bergmann.Development of the Varioscope AR: A See-through HMDfor Computer-Aided Surgery. In Proceedings InternationalSymposium Augmented Reality - ISAR, pages 47–53, 2000.

[5] M. Blackwell, A. DiGioia, and C. Nikou. Image OverlayTechniques for Surgery. Transactions on Medical ImageAnalysis, June 1998.

[6] M. Blackwell, C. Nikou, A. DiGioia, and T. Kanade. An Im-age Overlay system for medical visualization. Transactionson Medical Image Analysis, 4:67–72, 2000.

[7] W. E. L. Grimson, G. J. Ettinger, et al. Evaluating and vali-dating an automated registration system for enhanced realityvisualization in surgery. Proceedings of Computer Vision,Virtual Reality, and Robotics in Medicine ’95, pages 3–12,1995.

[8] H. Kato, M. Billinghurst, R. Blanding, and R. May. AR-ToolKit PC Version 2.11, 1999.

[9] W. Lorensen, H. Cline, et al. Enhancing Reality in the Op-erating Room. Proceedings of the 1993 IEEE VisualizationConference, pages 410–415, 1993.

[10] Medarpa Consortium. MEDARPA, MEDical AugmentedReality for the PAtient. http://www.medarpa.de, 2003.

[11] J. P. Mellor. Enhanced Reality Visualization in a SurgicalEnvironment. AI Lab . Cambridge, MA, Massachusetts In-stitute of Technology, page 102, 1995.

[12] S. Nakajima, K. Nakamura, K. Masamune, I. Sakuma, andT.Dohi. Three-dimensional medical imaging display withcomputer-generated integral photography. Comput MedImaging Graph, (25(3)):235–241, 2001.

[13] F. C. Park and B. Martin. Robot sensor calibration: Solvingax=xb on the euclidean group. volume 10, pages 717–721,October 1994.

[14] G. Reitmayr and D. Schmalstieg. An open software archi-tecture for virtual reality interaction. In ACM Symposium onVirtual Reality Software & Technology 2001, 2001.

[15] M. Ribo, M. Brandner, and A. Pinz. A Flexible SoftwareArchitecture for Hybrid Tracking. In Workshop on Integra-tion of Vision and Inertial Sensors of the 11th InternationalConference on Advanced Robotics (ICAR) 2003, Coimbra,Portugal, June 2003.

[16] M. Sandin, B. Bruegge, G. Klinker, A. MacWilliams, T. Re-icher, S. Riß, C. Sandor, and M. Wagner. Design of aComponent-Based Augmented Reality Framework. IEEEand ACM International Symposium on Augmented Reality(ISAR),New York,NY, October 29-30 2001.

[17] B. Schwald and C. Malerzcyk. Controlling Virtual WorldsUsing Interaction Spheres. In C. Vidal, editor, 5th Sym-posium on Virtual Reality (SVR), Fortaleza, Brazil, 2002.Brazilian Computer Society.

[18] B. Schwald and H. Seibert. Registration Tasks for a Hy-brid Tracking System for Medical Augmented Reality. Inaccepted for the International Conference on ComputerGraphics, Visualization and Computer Vision (WSCG 2004),2004.

[19] F. Seibert. Stereo-based Augmented Reality in Medicine.3. German-Korea Joint Conference on Advanced MedicalImage Processing, EWHA University, Seoul, Korea, August1998.

[20] O. Tonet, G. Megali, S. D’Attanasio, P.Dario, M. Carrozza,M. Maracci, S. Martelli, and P. L. Palombara. An Aug-mented Reality Navigation System for Computer AssistedArthroscopic Surgery of the Knee. Lecture Notes in Com-puter Science, S.L. Delp, A.M. DiGioia and B. Jaramaz Eds.,Springer, 1935:1158–1162 (MICCAI 2000), October 2000.

[21] M. Tuceryan and N. Navab. Single Point Active AlignmentMethod (SPAAM) for Optical See-through HMD Calibra-tion for AR. In Proceedings of ISAR 2000, pages 149–158,2000.

[22] G. Welch and G. Bishop. An Introduction to the KalmanFilter. Computer science technical report 95-041, UNC-CH,1995.

[23] R. Yamanaka, K. Yamamoto, N. Handa, and H. Yoshikura.A 3D Display with a Linearly Moving Mirror to Reflect aSeries of 3D Cross Sections and Its Application to Non-invasive Angiography. Transactions on medical Imaging,7(3):193–197, September 1988.

Proceedings of the Computer Graphics International (CGI’04)

1530-1052/04 $20.00 © 2004 IEEE