From BIM to VRIntegrating immersive visualizations in the current design process

Mikael Johansson1, Mattias Roupé2, Mikael Viklund Tallgren31,2,3Chalmers University of Technology, Sweden

1,2,3{jomi|roupe|mikael.tallgren}@chalmers.se

This paper presents a system that allows immersive visualizations to become anatural and integrated part of the current building design process. It is realizedthrough three main components: (1) the Oculus Rift - a new type of HeadMounted Display (HMD) directed at the consumer market, (2) a real-timerendering engine supporting large Building Information Models (BIM) that is, (3)implemented as a plug-in in a BIM authoring software. In addition to providedetails regarding the implementation and integration of the different componentsin our system, we present an evaluation of it from three different perspectives;rendering performance, navigation interface and the ability to support fast designiterations.

Keywords: Building Information Modeling, BIM, Virtual Reality, Real-timerendering, HMD

INTRODUCTIONDuring the design process of a building, it is im-portant that all the involved actors understand, par-ticipate, communicate, and collaborate with eachother to obtain a high quality outcome of the de-sign process. Hall and Tewdwr-Jones (2010) highlightthe communication difficulties between the differ-ent stakeholders in the design and planning process.Communication difficulties mainly occur as a resultof the different planning cultures, and because thereis insufficient collaboration and information sharingduring the process. The most common problem isthat the information is not presented in such a waythat people can understand it.

In this context, real-time visualizations and Vir-tual Reality (VR) have been shown to offer an effi-

cient communication platform (Bouchlaghem et al.,2005). With the ability to navigate freely through3D scenes from a first-person perspective, it is possi-ble to present and communicate ideas regarding fu-ture buildings in a way that facilitates understandingamong all involved parties, despite their backgroundor professional expertise. While the use of this tech-nology has been naturally limited in the past due tolack of available 3D data from the design process, therecent introduction of Building Information Models(BIM) within the AEC field has opened up new pos-sibilities. With the use of BIM the required 3D datacan be extracted from the architect's own design-environment, instead of creating it from scratch us-ing 2D-plans, elevations and sketches as a reference.Because of this, use of real-time visualizations has be-

Contribution 195 (Preprint) - figure and table placement subject to change- eCAADe 32 | 1

come more accessible in practice.To further enhance user experience it is com-

monly advocated to take advantage of immersivedisplay technologies. Although real-time visualiza-tions have been shown to be useful per se, stere-oscopy, large screen and wide field of view all pro-vide additional benefits. When comparing a non-immersive (monitor) solution to a four-screen (3wallsand a floor) CAVE solution, Shiratuddin et al. (2004)found consistently higher ratings for the latter re-garding level of realism, ease of navigation, sense ofscale and overall suitability for design and decision-making tasks.

In this context, Head Mounted Displays (HMD)also represents a viable option. Still, as avail-able alternatives (until very recently) have been ei-ther low-cost-low-performance or high-cost-high-performance devices (Dörner et al., 2011), CAVEs andPowerWalls have emerged as the de facto standardwhen it comes to immersive visualizations. Whenconsidering practical applications, these types of so-lutions have been shown beneficial during the de-sign of hospital patient rooms and courtrooms, aswell as for design review sessions in general (Cas-tronovo et al., 2013).

However, when considering the integration anduse of immersive VRwithin the actual design process,the current adaptation in the AEC field still suffersfrom a number of limitations:

• High cost: Even if the cost of display and PChardware has been rapidly decreasing overthe past years, fully or semi-immersive solu-tions such as CAVEs or PowerWalls are still ex-pensive (DeFanti et al., 2011).

• Limited accessibility: Regardless of dis-play technology (e.g. immersive or semi-immersive), the use of a specific room or stu-diowill naturally restrict visualization sessionsto a single location. Even if situated close tothe designers working environment it makesthe use of VR less accessible, both physicallyand mentally. This immobility has also been

reported inconvenient for clients and otherstakeholders (Sunesson et al., 2008).

• LimitedBIM-support: Even if createdwith vi-sualization in mind, real-time constraints andstereo rendering often require the input 3Ddata to be further optimized in order to befully functional in the VR environment. Whenconsidering BIMs this process typically be-comes even worse due to a large number ofindividual objects and high geometric com-plexity (Dalton and Parfitt, 2013). In addition,manyBIMauthoring applicationshave limitedor missing support for materials and texturedefinitions when exporting 3D-data for visu-alization purposes (Kumar et al., 2011).

In this paper we present a solution that overcomesthe above mentioned limitations and allows immer-sive VR to become a natural and integrated part ofthe design process. It is realized through three maincomponents: (1) the Oculus Rift Head Mounted Dis-play (HMD) - a comparably low cost device that sup-ports a large field of view, stereoscopic viewing andphysically rotation, (2) an efficient real-time render-ingengine supporting large3Ddatasets that is (3) im-plemented as a plug-in in a BIM authoring software.

APORTABLE SYSTEMFOR IMMERSIVE BIMVISUALIZATIONFigure 1 shows the different components of our pro-posed system: The Oculus Rift HMD, the real-timeviewer application implemented as a plug-in in RevitArchitecture and a so-called PowerPoint remote con-trol used as a navigation interface, all connected to alightweight laptop. In the following subsections wepresent and discuss these components in more de-tail.

TheOculus Rift HMDIn order to provide an immersive visualization en-vironment our proposed system takes advantage ofthe Oculus Rift HMD. The Rift is a new affordable (ex-pected price range $300-$350) virtual reality device

2 | eCAADe 32 - Contribution 195 (Preprint) - figure and table placement subject to change

Figure 1System overview;the Oculus HMD,the Revit Viewerplug-in and thePowerPoint remotecontrol

directed at the consumer's market, mainly to provideimmersive experiences for videogames. Althoughcurrently only available in the formof aDeveloper Kit,it is expected to be available on the broad consumermarket during 2015. The device provides approxi-mately 100° field of view, stereoscopic 3D view andincludes a gyroscope, an accelerometer and a mag-netometer to determine the orientation of the user'shead in the real world.

As with any other stereo-providing display solu-tion the 3d scene has to be rendered twice, once foreach eye. In the case of the Rift this is implementedby means of split-screen rendering, where the lefthalf of the screen corresponds to the left eye, and viceversa. With a full-screen resolution of 1280 x 800 pix-els, this gives an effective resolution of 640 x 800 pereye.

However, although this approach to supportstereo vision is conceptually simple, the actual ren-dering process is a bit more involved. Due to thelenses, which provide an increased field of view, apincushion distortion of the image is introduced. Tocancel out this effect, the rendering has to be doneat a higher resolution, followed by a post-processingstep that performs a barrel distortion. Preferably, an-tialiasing should also be enabled, as this greatly en-hances the image quality.

So, in effect, even if the devices' resolution is"only" 1280 x 800 pixels, the actual rendering (rasteri-

zation) has to be performed at a much higher resolu-tion (i.e. 2194 x 1371) , potentially with antialiasingenabled, followed by a full-screen post-processingstep. Obviously, these requirements put addition-ally stress on the graphics hardware, which, in turn,put high demands on a rendering engine to deliverenough rendering performance to support an inter-active experience.

The rendering engineAn important property for any type of real-time visu-alisation system is its ability to maintain a sufficientlyhigh frame rate. For typical desktop applications (i.e.non-immersive) 15Hz is often considered aminimum(Yoon et al. 2008), although 30 or 60 Hz is generallyadvocated in order to provide a satisfactory level ofinteractivity. However, for HMDs, such as the Ocu-lus Rift, the minimum interactivity-demands are typ-ically higher, as physical interaction and display up-date becomes much more integrated. Ultimately, auser's headmovement should directly correspond toan update of the display in order to reduce the risk ofpotential conflicts between visual-vestibular sensory.In this context a minimum frame rate of 60 Hz is of-ten recommended (Adelstein et al., 2003), althoughhigher values have alsobeenproposed (Jerald, 2010).

When considering that the task of visualizingBIMs interactively is known to be a challenge in it-self (Steel et al., 2012; Johansson and Roupé, 2012),

Contribution 195 (Preprint) - figure and table placement subject to change- eCAADe 32 | 3

Figure 2The viewer plug-ininterface in Revit

the frame rate requirements posed by using an HMDthusput veryhighdemandson the renderingengine.This, especially, as the 3D dataset has to be renderedtwice every frame in order to support stereoscopic vi-sion, followed by a full-screen post-processing step.

To address these requirements we have devel-oped an efficient rendering engine that takes advan-tage of two characteristics shared by typical buildingmodels - high level of occlusion and frequent reuseof identical building components. The engine, whichis described in more detail in (Johansson and Roupé,2012) and (Johansson, 2013), uses an efficient occlu-sion culling algorithm to restrict rendering efforts tovisible objects only, and takes advantageof hardwareinstancing to render replicated building componentsefficiently. These two acceleration techniques com-plement each other and are essential in order to fulfilthe requirements in terms of interactivity. However,we do not primarily use this rendering engine in aseparate application. To support an integrated de-sign environmentwe have instead implemented it asa viewer plug-in in Autodesk Revit.

The Revit plug-inFigure 2 illustrates how our viewer plug-in is inte-grated in Revit from a user's perspective. With a BIMloaded in Revit the viewer is initialized from the Add-Ins tab, resulting in the real-time 3D visualization rep-resentation becoming visible in a newwindow. Afterthat a user is free to either navigate the model in atypical desktop fashion using mouse and keyboard,or connect the Oculus HMD to experience themodelimmersively.

From a programmers point of view the plug-in extracts the required 3D data through the Re-vit C# API, which exposes the entire underlying BIMdatabase. To speed-up the data extraction processand to keep the memory footprint low, we take ad-vantage of geometry instancing (i.e. that severalidentical components can share the same geomet-rical representation), which is an integral part of theinternal Revit database. Every time a unique geomet-ric representation is encountered for the first time, allof its data is extracted. For all subsequent cases thepreviously extracted geometry data is used in combi-nation with a unique transformation.

4 | eCAADe 32 - Contribution 195 (Preprint) - figure and table placement subject to change

As the rendering engine and the Oculus API iswritten in C++, the different software componentsneeds to be connected through a C++/Cli bridge(Heege, 2007). The complete architecture is illus-trated in Figure 3 and also shows how the GUI-module is separated from the actual plug-in, essen-tially allowing us to run the viewer as a standaloneapplication on a system without Revit installed, withidentical interface.

Figure 3System architecture

Previous versions of the Revit API did not exposematerial data such as colors and textures, making itvery difficult to use BIMs, without further treatment,for visualization sessions related to aesthetics. Fortu-nately, since version 2014, theAPI has been extendedwith a Custom Export API, that facilitates the extrac-tion ofmaterial and texture data aswell as texture co-ordinates. Because of this, it is now possible to ex-tract complete visualization models, with materialsand textures assigned, directly from the BIM author-ing software.

The navigation interfaceThe use of an HMD makes traditional navigation in-terfaces, such as ones with keyboard and a mouseharder to master. As the user cannot see anythingin the real world, even seemingly simple tasks, suchas pressing a specific key on the keyboard or evengrabbing the mouse becomes much more involved.For very experienced users that daily works with, and

navigates in, 3D models this does not necessarilypose itself as aproblem, however, for peoplewith lessexperience it can easily become a huge obstacle.

In order to allow for any type of user we havetherefore developed a very simple navigation inter-face bymeans of a so-called PowerPoint remote con-trol. As illustrated in Figure 4, a user can move for-ward or back by pressing the corresponding buttonson the remote control, with the direction of move-ment being decided by the user's orientation of thehead.

SYSTEM EVALUATIONTo illustrate the effectiveness of our proposed systemwe present an evaluation of it from three differentperspectives - rendering performance, navigation in-terface and the ability to support fast design itera-tions. As test-model we have used a BIM receivedfrom a real-world project, a ten-story office buildingwhich is currently being built in Gothenburg, Swe-den (Figure 2). The model is primarily an architec-tural model, with noMechanical, Electrical or Plumb-ing (MEP) data present, however it does contain fur-niture and other interior equipment (See Figure 5).The model was created in Revit Architecture 2013and contains approximately 4,400,000 triangles, dis-tributed over 15,000 individual objects.

Figure 4The navigationinterface

Contribution 195 (Preprint) - figure and table placement subject to change- eCAADe 32 | 5

Figure 5Interior viewpointat the third level ofthe test-model

Rendering performanceThe rendering performance test was performed ontwo different computers, one workstation and onelaptop. The workstation was equipped with an Inteli7 3.06 GHz CPU, 6 GB of RAM and an Nvidia GeForceGTX 570 GPU running Windows 7 x64. The laptopwas equipped with an Intel i7 1.9 GHz CPU, 4 GB ofRAM and an Nvidia GeForce GT 620M GPU runningWindows 8 x64. On both system, two different cam-era paths was used; one interior at the third floor ofthe building, and one exterior at the ground-level infront of the building. The results from these tests arepresented in Figure 6. To better illustrate the per-formance gain offered by our rendering engine wealsopresentperformance results obtainedwhenonlyview frustum culling is enabled (i.e. only discardingobjects that are outside the cameras view frustum).The followingabbreviations areused (andcombined)in the plots: OC forOcclusionCulling, HI for HardwareInstancing, VFC for View Frustum Culling, and MSAAfor 4x MultiSample AntiAliasing.

As can be seen in the plots, the use of additionalacceleration techniques is vital in order to providethe required level of interactivity. Withonly view frus-tum culling enabled (VFC) it becomes very difficult toguarantee a minimum frame rate of 60 Hz, even onthe workstation system. In fact, for the given camerapaths, not even 20Hz canbeguaranteedonboth sys-tems.

However, with the combined use of occlusionculling (OC) and hardware instancing (HI) it is possi-ble to fulfil the interactivity demands. The only ex-

ception appears during parts of both camera pathson the laptop system when antialiasing is enabled.Although not by much (the lowest recorded framerates are 52 Hz and 48 Hz, respectively) it is defi-nitely below our target frame rate of 60 Hz. Still,as antialiasing-capacity scale well with GPU perfor-mance (which is not necessarily the case with 3Dmodel complexity due to driver overhead), we ex-pect this particular issue to be solved by increasinglyfaster GPUs. This, especially when also consideringthat these tests were performed on a, at the time ofwriting, two year old laptop.

Nevertheless, without antialiasing activated, ourrendering engine canprovide the required level of in-teractivity, even on a lightweight laptop system.

Navigation interfaceAs part of a different, but related, research projectwe have performed an initial evaluation of the nav-igation interface with members of the on-site teamthat is currently erecting the real building. This groupof people included the site manager as well as fiveconstruction workers from different sub-trades (pip-ing, ventilation, sprinklers, prefab and electrical). Noone, except for the site manager, had any previousexperience from working with, or navigating in, 3D-models. While freely navigating and inspecting thedigital representation of the building that they werecurrently erecting, theywere askedquestions regard-inghow they felt that this typeof interface couldhelpthem extract information to support their daily workand what additional features they would like the sys-tem to have. Except for the electrical trade worker,they all expressed that this type of visual interfacehelped them to get a better understanding, not onlyin terms of specific details, but also for the project asa whole.

However, perhaps more interesting in this con-text, is the fact that we observed that all of them, in-cluding the electrical tradeworker, were able to navi-gate in themodel with ease. Based on our own previ-ous experience we know that this is typically not thecase when inexperienced people are faced with the

6 | eCAADe 32 - Contribution 195 (Preprint) - figure and table placement subject to change

Figure 6Frame rates for theexterior (top) andinterior (bottom)camera paths onthe workstation(left) and the laptop(right) system(OC=OcclusionCulling,HI=HardwareInstancing,MSAA=4xMultiSampleAntiAliasing,VFC=View FrustumCulling).

task of navigating in a 3D-model using the keyboardand a mouse (i.e. mouse-look and WASD).

Design iterationsThe benefit of having the visualization environmentclosely connected to the BIM authoring environmentbecomes especially clear when considering rapid de-sign iterations. To illustrate this we will provide twoconcrete examples applied to our test-model: one isthe change of window types on one of the facadesand theother is the removal of two conference roomson the third floor in order to extend the office land-scape area.

Although solutions have been proposed whereit is possible to modify architectural models directlyin an immersive environment (Schulze et al., 2014),these systems typically only support insertion andrepositioning of pre-made objects or creation and

modification of simple geometry. In contrast, ourexamples are much more involved, as they includeoperations on fairly complex objects that also affectother objects. For instance, when changingwindowsto a type that is geometrically smaller or larger, thegeometry for the host object, the wall, needs to berecomputed in order for the opening size to matchthe correspondingwindow size. Although such func-tionality would have been technically possible toimplement in our viewer, we have instead focusedon making the "conversion" from design-model tovisualization-model as fast as possible. In the case ofour test model, this process takes approximately 20seconds. That is, regardless of modification, the onlytime needed to produce a new version of the immer-sive visualization will be the time required to makethe actualmodifications in Revit, plus 20 seconds. Forthe examples described above this time corresponds

Contribution 195 (Preprint) - figure and table placement subject to change- eCAADe 32 | 7

Figure 7Before (left) andafter (right) rapiddesignmodifications. Toprow illustratesremoval ofconference room.Bottom rowillustrates change ofwindow types.

to 3 minutes and 2.5 minutes, respectively. In Figure7, these twomodifications are illustratedwith "beforeand after" screenshots.

CONCLUSIONS AND FUTUREWORKWehavepresenteda systemthat allows immersive vi-sualization to become a natural and integrated partof the building design process. By using the Ocu-lus Rift HMD we are able to provide an immersive vi-sualization environment without the need of a ded-icated facility to host a PowerWall or CAVE installa-tion. In addition to greatly reduce investment costs,this feature also makes the use of VR within a projectbecome physically more accessible. As the technol-ogy is portable, clients and design team memberscan take advantage of immersive visualization ses-sionswithout the need to travel to a specific location.

To further address accessibility, we have devel-oped a rendering engine capable of managing largeand complex 3D datasets in real-time. As a resultwe can directly visualize large and complex BIMs, instereo,without theneed tomanually optimizeor pre-pare the input dataset. To support an integrated de-sign environment this rendering engine has been im-plemented as a viewer plug-in in Autodesk Revit. Be-

cause of this, immersive design review sessions canbe performed directly in the BIM authoring softwarewithout the need to export any data or create a sep-arate visualization model.

In addition, we have presented an initial evalu-ation of the proposed system with a BIM receivedfrom a real-world project. Regarding rendering per-formance, navigation interface and the ability to sup-port fast design iterations, we have shown that it hasall the needed properties in order to function well inpractice.

For future work we are considering several dif-ferent directions, including studies related to spatialunderstanding with HMDs, enhancement of the in-teraction interface, investigation of benefits with oursystem in different contexts (i.e. design review, plan-ning, on-site information extraction, etc.) as well asfurther research to improve rendering performance.

REFERENCESAdelstein, BD, Lee, TG and Ellis, SR 2003 'Head

tracking latency in virtual environments: psy-chophysics and amodel', In Proceedings of theHu-man Factors and Ergonomics Society AnnualMeet-ing, pp. 2083-2087

8 | eCAADe 32 - Contribution 195 (Preprint) - figure and table placement subject to change

Bouchlaghem, D, Shang, H, Whyte, J and Ganah, A2005, 'Visualisation in architecture, engineeringand construction (AEC)', Automation in Construc-tion, 14, pp. 287-295

Castronovo, F, Nikolic, D, Liu, Y and Messner, JI 2013'An evaluation of immersive virtual reality sys-tems for design reviews', Proceedings of the 13thInternational Conference on Construction Applica-tions of Virtual Reality

Dalton, B and Parfitt, M 2013 'Immersive Visualizationof Building Information Models', Design Innova-tionResearchCenterworkingpaper6, University ofReading, UK

DeFanti, T, Acevedo, D, Ainsworth, RA, Brown, MD,Cutchin, S, Dawe, G, Doerr, KU and Johnson, A2011, 'The Future of the CAVE', Central EuropeanJournal of Engineering, 1, pp. 16-37

Dörner, R, Lok, B and Broll, W 2011, 'Social Gamingand Learning Applications: A Driving Force forthe Future of Virtual and Augmented Reality?', inCoquillart, S (eds) 2011, Virtual Realities, Springer,pp. 51-76

Hall, P and Tewdwr-Jones, M 2010, Urban and Re-gional Planning, Routledge

Heege,M2007, ExpertC++/CLI:NET forVisualC++Pro-grammers, Apress

Jerald, JJ 2010, Scene-motion-and latency-perceptionthresholds for head-mounted displays, Ph.D. The-sis, University of North Carolina

Johansson, M 2013 'Integrating Occlusion Cullingand Hardware Instancing for Efficient Real-TimeRendering of Building Information Models', Inter-national Conference on Computer Graphics Theoryand Applications (GRAPP 2013), pp. 197-206

Johansson, M and Roupé, M 2012 'Real-Time Ren-dering of Large Building Information Models',CAADRIA 2012 - Beyond Codes & Pixels, pp. 647-656

Kumar, S, Hedrick, M, Wiacek, C and Messner, JI2011, 'Developing an experienced-based designreviewapplication for healthcare facilities using a3d game engine', Journal of Information Technol-ogy in Construction (ITcon), 16, pp. 85-104

Schulze, JP, Hughes, CE, Zhang, L, Edelstein, E andMacagno, E 2014 'CaveCAD: a tool for architec-tural design in immersive virtual environments',Proceedings of SPIE 9012, The Engineering Realityof Virtual Reality 2014

Shiratuddin, MF, Thabet, W and Bowman, D 2004'Evaluating the effectiveness of virtual environ-ment displays for reviewing construction 3Dmodels', Proceedings of CONVR 2004, pp. 87-98

Steel, J, Drogemuller, R and Toth, B 2012, 'Model in-teroperability in building informationmodelling',Software & SystemsModeling, 11, pp. 99-109

Sunesson, K, Allwood, CM, Paulin, D, Heldal, I, Roupé,M, Johansson, M and Westerdahl, B 2008, 'VirtualReality as aNewTool in theCity PlanningProcess',Tsinghua Science & Technology, 13, pp. 255-260

Yoon, SE, Gobbetti, E, Kasik, D and Manocha, D 2008,Real-Time Massive Model Rendering, Morgan &Claypool Publisher

Contribution 195 (Preprint) - figure and table placement subject to change- eCAADe 32 | 9