Master of Science in Game and Software EngineeringMay 2020

Run-time Terrain Deformation inUnreal Engine 4

Rikard Magnom

Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technology inpartial fulfilment of the requirements for the degree of Master of Science in Game and SoftwareEngineering. The thesis is equivalent to 20 weeks of full time studies.

The authors declare that they are the sole authors of this thesis and that they have not usedany sources other than those listed in the bibliography and identified as references. They furtherdeclare that they have not submitted this thesis at any other institution to obtain a degree.

Contact Information:Author(s):Rikard MagnomE-mail: rima15@student.bth.se

University advisor:Adjunct Prof. Stefan PeterssonDepartment of Computer Science

Faculty of Computing Internet : www.bth.seBlekinge Institute of Technology Phone : +46 455 38 50 00SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

Abstract

Background. When developing video games, an often desired trait is that of playerimmersion. Player immersion can be achieved through many different ways. How-ever, a common technique used is to allow the player to affect the environment insome way, through destruction or deformation. Unreal Engine 4 is a commonly usedcommercial game engine, that lacks features necessary to deform terrain at run-time.Objectives. This thesis explores the viability of run-time performant terrain defor-mation in Unreal Engine version 4.23. A deformation technique of a depth-basedDynamically-Displaced Height Map (DDHM) is selected for implementation andtested with various sizes of landscape terrains.Methods. A small literature study is performed to select a feasible run-time terraindeformation technique. The technique is then implemented in the engine, along withmodifications to the engine to provide the necessary functionality. The implementa-tion is then tested on performance and accuracy.Results. The DDHM technique is shown to provide run-time performant terrain de-formation for smaller terrain sizes, consisting of an individual landscape component.The selected technique does not meet the run-time requirements for full, large-scaledeformation.Conclusions. Run-time performant terrain deformation is shown to be achievablein Unreal Engine 4.23. While the selected technique fails to achieve satisfying resultsfor larger terrain sizes, results on individual landscape components show feasibilityfor generic technique implementations.

Keywords: terrain, deformation, real-time, unreal engine

i

Sammanfattning

Bakgrund. Under utvecklingen av datorspel är det ofta en eftertraktad egenskapatt kunna övertyga spelaren om realismen i spelvärlden. Detta kan uppnås på flerasätt, men en vanlig teknik är att låta spela påverka omgivningen i spelet på någotsätt, genom förstörelse eller deformering. Unreal Engine 4 är en vida använd kom-mersiell spelmotor som saknar funktionalitet för att deformera terräng i realtid.Syfte. Syftet med detta arbete är att utforska genomförbarheten i att tillföra ter-rängdeformering i realtid i Unreal Engine version 4.23. En deformeringsteknik in-volverande en “Dynamically-Displaced Height Map” (DDHM) väljs ut för implemen-tation och testas med olika storlekar på landskapsterräng.Metod. En mindre litteraturstudie genomförs för att välja en rimlig teknik för ter-rängdeformering i körtid. Tekniken implementeras sedan i motorn, i sambard medmodifikationer till motorn för att tillföra nödvändig funktionalitet. Implementatio-nen testas sedan på prestanda och precision.Resultat. DDHM-tekniken visas kunna tillföra körtidsduglig terrängdeformering förmindre terrängstorlekar, bestående av individuella landskapskomponenter. Teknikenmöter inte realtidskriterierna för full, storskalig terrängdeformation.Slutsatser. Deformering av terräng i körtid visas vara genomförbart i Unreal Engine4.23. Den utvalda tekniken visar lovande resultat för mindre storlekar av landskap-sterräng, men misslyckas att prestera för större terrängstorlekar.

Nyckelord: terräng, deformering, realtid, unreal engine

iii

Acknowledgments

First and foremost I would like to sincerely thank Stefan Petersson for his superbguidance and valuable feedback during the writing of this thesis, and for helping mestay sane throughout the process. I also express great gratitude towards Logic Artistsfor collaborating with me in the execution of this thesis, and for giving me free reignto work whenever needed throughout the process. An additional special thank youto my dear friend James Partridge for his assistance in proofreading. Finally, a warmthank you to my mother and my grandparents, for staying supportive of me foreverand always.

v

Contents

Abstract i

Sammanfattning iii

Acknowledgments v

1 Introduction 11.1 Environment Interaction . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Objectives and Research Questions . . . . . . . . . . . . . . . . . . . 11.3 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Related Work 32.1 Hardware Tessellation . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Unreal Engine Landscape Systems . . . . . . . . . . . . . . . . . . . . 42.3 Displacement mapping . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4 Parallax mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Method 73.1 Literature Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Depth-based DDHM . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.3 Visual Representation . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Implementation 134.1 Deformation Actor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.1.1 Actor Initialization . . . . . . . . . . . . . . . . . . . . . . . . 134.1.2 Class Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2 Engine Source Code Modifications . . . . . . . . . . . . . . . . . . . . 154.3 Engine Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.3.1 Limitations of the USceneCaptureComponent2D . . . . . . . . 184.3.2 Limitations of the Landscape Module . . . . . . . . . . . . . . 18

5 Results 215.1 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.1.1 Testing Environment . . . . . . . . . . . . . . . . . . . . . . . 215.1.2 Full Algorithm Performance . . . . . . . . . . . . . . . . . . . 225.1.3 Render Target Import Performance . . . . . . . . . . . . . . . 235.1.4 Heightmap Update Performance . . . . . . . . . . . . . . . . . 235.1.5 Average Error . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

vii

5.1.6 Conversion Resolution Scalar . . . . . . . . . . . . . . . . . . 24

6 Analysis and Discussion 276.1 DDHM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.2 Generic Run-time Deformation . . . . . . . . . . . . . . . . . . . . . 276.3 Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.4 Validity Threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7 Conclusions 29

8 Future Work 31

References 33

A Deformation Actor Source Code 37

viii

List of Figures

2.1 Visual example of overlapping landscape components. Note the du-plicated vertices [13]. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 Parallax mapping used on a wall texture in Bethesdas The ElderScrolls IV: Oblivion [28]. . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.1 Details of the first Google Scholar database search of the small liter-ature study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2 Details of the first BTH Summon database search of the small litera-ture study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Details of the second BTH Summon database search of the small lit-erature study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Details of the third BTH Summon database search of the small liter-ature study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.5 The depth capture camera configuration for generation of a DDHM [1]. 103.6 The process of depth information being converted to a heightmap

through the DDHM technique. (a) shows the object depth clippingthrough the terrain and being rendered to the depth capture rendertarget. (b) shows the depth value data being used to repopulate theheight map. [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.7 Example of one iteration of the terrain deformation. (a) shows theoriginal unmodified terrain. (b) shows the sphere mesh used for thisdeformation example, rendered for demonstration purposes. (c) showsthe resulting deformation in the terrain. . . . . . . . . . . . . . . . . 11

3.8 Example of height levels converging to set stages as a result of colourbanding in the depth buffer. This example shows the result on thelandscape after 30 iterations. . . . . . . . . . . . . . . . . . . . . . . . 12

4.1 One iteration of the implementation performed on a flat landscape,visualized in gray, deforming around a cube shaped mesh, visualizedin orange. (a) shows the configuration before the deformation. (b)shows the result of one iteration of the implementation. . . . . . . . . 14

4.2 The orthographic projection matrix. The direction parameters left,right, top, bottom represent the size of the orthographic projectionfrom the center. The parameters far and near represent the distancefrom the camera of the far and near planes respectively. . . . . . . . . 18

5.1 Execution time of the algorithms performed on a single landscapecomponent with no collision updating. . . . . . . . . . . . . . . . . . 22

ix

5.2 Average error distance per vertex with the DepthCaptureCamera placedwith the near plane at various distances from the landscape mesh. . . 24

x

List of Tables

4.1 All modified engine source code files. . . . . . . . . . . . . . . . . . . 17

5.1 The results of the execution time of a full iteration with collision up-dating enabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.2 The results of the execution time of a full iteration with collision up-dating disabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5.3 The results of the execution time of the render target import. . . . . 235.4 The results of the execution time of Algorithm 3 with collision updat-

ing enabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.5 The results of the execution time of Algorithm 3 with collision updat-

ing disabled. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.6 Depth buffer heightmap resolution and resolution conversion scalar at

various distances of the DepthCaptureCamera. . . . . . . . . . . . . . 25

xi

List of Algorithms

1 ImportHeightmapFromRenderTarget . . . . . . . . . . . . . . . . . . . 152 DeformActorInitialize . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 UpdateLandscapeHeightData . . . . . . . . . . . . . . . . . . . . . . . 17

xiii

Chapter 1

Introduction

In video game development, an often desired element is that of player immersion.Player immersion, which involves convincing the player of the realism of the inter-active game world they are experiencing. A strong contributor to player immersioninvolves being able to directly interact and affect the game environment. This can bedone a multitude of ways such as simply being able to have the environment respondto player interaction [21].

1.1 Environment Interaction

One common form of environment interaction found in video games is the ability forthe player to destroy parts of the environment dynamically. Destructible environ-ments in games have a long history, dating back to early examples such as the 1978arcade game Space Invaders, which featured cover for the player to hide behind.This cover would then progressively get destroyed upon being hit [3]. This trendcontinued into the 3D era of video games, with series such as Volition’s Red Factionutilizing interactive and fully destructive environments as a primary selling point.Utilizing their “GeoMod technology” the player could dynamically destroy parts ofthe cohesive environment. This also included being able to modify the terrain inaddition to the placed game objects, introducing further interactability [31].

While plenty of work exists exploring destructible terrain in video games in real-time, from volume-preserving techniques [24] to more specific applications, such ascreating vehicle trails [10] and tire tracks [4], not much work exists exploring solutionsbased in Unreal Engine 4. Existing research is primarily limited to work outside ofcommercial game engines or in the cases where game engines are used, the Unitygame engine by Unity Technologies is usually the engine of choice, which has adifferent terrain and landscape system compared to Unreal Engine 4.

1.2 Objectives and Research Questions

This thesis aims to investigate the feasibility of run-time terrain deformation byan arbitrary mesh given the existing terrain and landscape systems present in EpicGames’ Unreal Engine version 4.23 game engine. A suitable technique for run-timeterrain deformation is to be identified through a small literature study. The identifiedtechnique will then be implemented in Unreal Engine 4.23 and will be quantitativelyevaluated on the execution time of the algorithm, measured in milliseconds. If the

1

2 Chapter 1. Introduction

specific method selected contains additional steps that would not apply in the genericcase of terrain deformation, this will be evaluated separately. The accuracy of thedeformation method will be measured in Unreal Units, the primary unit of measure-ment in 3D-space in the Unreal Engine game engine [14].

This thesis is performed on behalf of Logic Artists, who have requested the im-plementation to be performed in version 4.23 of Unreal Engine. As of the writing ofthis thesis, no newer version of the engine exists that contains major modificationsto any module required for the implementation presented in Chapter 4.

The research questions posed in this thesis are:

1. Does a run-time terrain deformation technique for arbitrary mesh deformationexist that can be implemented in Unreal Engine 4.23?

2. How well can we achieve run-time performant terrain deformation in UnrealEngine 4.23?

The criteria for concluding whether the implementation of the identified techniqueis suitable for run-time utilization is that the method needs to be suitable for usein a game environment where the frame rate does not drop below 30 frames persecond. This criterion is necessary to not negatively impact the gameplay experienceof a player, should this technique be implemented in a computer game [5]. Theimplementation will be tested on a platform of equal or comparable performance tothat of the average consumer configuration as recorded by a hardware survey at thetime of writing this thesis [7].

1.3 TerminologyThis section defines some important terminology used throughout the thesis.

DDHM - Dynamically Displaced Height-Mapdepth-based - Primarily utilizing the depth-pass of the rendering pipelineUE - Unreal EngineUnreal Unit - Unit of measurement used in Unreal EngineActor - Base component in a scene in Unreal EngineLandscape Component - A single component of a terrain Landscape in UnrealEngineRun-time - In this thesis, defined in Section 1.2 as being able to execute at 30frames per second

Chapter 2

Related Work

This chapter details various works relevant to the area of landscape and terraindeformation as well as covers the details of rendering techniques that can be used asa basis for run-time mesh deformation.

2.1 Hardware Tessellation

One method of achieving high-resolution meshes while keeping memory bandwidthlow is to utilize hardware tessellation. Hardware tessellation is a feature introducedinitially in the Direct3D 11 graphics API [26]. It allows for the generation of ad-ditional primitives at run-time through the use of patch primitives, giving a higherdetail representation of the mesh than a triangle-based approach [22]. Patches arefurther subdivided into triangles, points, and lines. In the Direct3D 11 API theseoperations are performed in the following three pipeline stages:

1. Programmable Hull shader stage

2. Fixed-function Tessellator stage

3. Programmable Domain shader stage

The programmable hull shader stage is performed once per patch primitive andoutputs a set of between 1 and 32 patch control points along with patch constantdata, which can be used by the domain shader stage.

Materials in Unreal Engine currently support two different types of tessellationthrough D3D11: Flat Tessellation and spline-based PN Triangles tessellation [30].The flat tessellation behaves like regular tessellation, meaning it simply subdividesprimitives to add geometric complexity. The spline-based PN Triangles tessellationmakes use of smoothing groups defined in the mesh to also smooth out sharp edges,which may be a desired feature when utilizing tessellation for adding detail to anobject. For displacement however, the more straight forward flat tessellation wouldbe more suitable as no unintended alterations will be made to the mesh.

One approach to the problem involves utilizing the 2D image space of a recti-linear grid of vertices. Aquilio A. et al. present a two-step render pass approachinvolving the use of a DDHM (Dynamically-Displaced Height Map) in conjunctionwith a generated offset map to create an incrementally deformed terrain structure[2]. The technique involves isolating the area of the terrain to be deformed throughan examination of the bounding boxes of the intersecting objects. This is followed

3

4 Chapter 2. Related Work

by positioning a camera perpendicularly to the ground plane directly below the areato be deformed. The first render pass is then executed using this camera, includingthe intersecting objects as well as the pre-deformed terrain, and finally storing theresulting depth values in a texture on the GPU. In the second render pass, this tex-ture is then sampled as an offset map for rendering the post-deformation terrain andthen stored as the DDHM for use in consecutive frames.

Figure 2.1: Visual example of overlapping landscape components. Note the dupli-cated vertices [13].

2.2 Unreal Engine Landscape Systems

Landscapes are a feature introduced in Unreal Engine 4 as a successor to the ter-rain feature of Unreal Engine 3. The landscape editor provides tools to create large,spanning terrains for use in games while also implementing several performance op-timizations. These optimizations include techniques such as dynamic level of detailor DLOD, meaning the resolution of the mesh varies depending on the distance fromthe viewer, as well as a subdivision of the terrain mesh into smaller, uniformly sizedcomponents. Each component is handled as a separate draw call by the engine andcan thus be regarded as it’s own individual mesh. Each component can optionallybe divided into four landscape sections. These sections are the base unit of LODcalculations in the engine and thus, the level of detail is constant throughout a singlelandscape section. As each component is independent of each other, the vertices ateach component edge overlap with the neighboring components as seen in Figure 2.1.

Landscapes in Unreal use a heightmap-based rendering technique. The landscapesare created as a uniform grid of vertices on a simple plane, which then samples atexture containing displacement values in the vertical axis to accurately positioneach vertex. This is an effective technique as displacements can be stored correctlyregardless of the resolution of the mesh. However, it also introduces some limitationssuch as the inability to accurately model non-vertical displacements such as caves.This is not necessarily a limitation as the concept of caves in a heightmap basedterrain could, for example, be circumvented by the use of transparency on the terrainin combination with a separate 3D-mesh representing the cave.

2.3. Displacement mapping 5

Layered materials are another key feature of landscapes. Material layering allowsfor up to ten separate materials to be painted onto landscape components and blendedthrough the use of blending techniques such as alpha or weight blending. The paintedweight-map, or splat-map, textures can be exported and imported individually andmay also contain additional information such as heightmap displacement, which canbe utilized for height blended terrain textures [12].

2.3 Displacement mappingDisplacement mapping is a technique that utilizes texture sampling to displace ver-tices on the GPU, allowing for detailed geometry to be rendered at various levels ofdetails without requiring explicit modeling of each level. As displacement mappingmodifies the positions of the vertices, it allows for a more accurate representation ofgeometric detail compared to using a technique such as normal mapping to simulatesurface detail through lighting.

Displacement mapping is primarily divided into two separate methods, intensitydisplacement, and vector displacement. Intensity displacement is the less memorydemanding of the two as the displacement map texture is used to store a singleintensity value, which is then sampled and used to displace each vertex along itsprecalculated normal. As intensity displacement only displaces vertices outwards,the result is a more smooth, continuous surface. Vector displacement instead storesdisplacement vectors in the texture, allowing vertices to be displaced in any directionbut at the cost of three times the memory per displacement map texture, as opposedto intensity displacement [29].

Figure 2.2: Parallax mapping used on a wall texture in Bethesdas The Elder ScrollsIV: Oblivion [28].

2.4 Parallax mappingParallax mapping, also known as “virtual displacement mapping” or “per-pixel dis-placement mapping,” is an alternative technique to traditional displacement map-ping. This technique involves adjusting the texture coordinates of pixels to give theillusion of depth [9] as seen in Figure 2.2. The geometry itself is therefore unaffectedsince this technique is performed entirely at the pixel shader stage after the geometryhas been rasterized. This is a benefit in terms of performance as it is independent of

6 Chapter 2. Related Work

the resolution of the initial mesh. However, a primitive application of the techniqueis unsuitable for large displacements such as terrain deformation and is primarilyused for minor detailing on textures [17].

Chapter 3

Method

This chapter details the small literature study conducted to select a suitable terraindeformation method and then presents the theoretical details of the selected method.

3.1 Literature Study

This section covers the small literature study performed with the purpose of selectinga suitable terrain deformation method to be implemented and tested in Unreal Engine4.23. The databases used for the small literature study are Google Scholar and theBlekinge Institute of Technology database Summon. Search results covering workinapplicable to real time rendering solutions in a game engine were discarded andnot considered.

Google Scholar Search 1Search Term real-time terrain deformationFilters Last 5 yearsDate searched 2020-03-04Results 9130Selected 1 [32]

Figure 3.1: Details of the first Google Scholar database search of the small literaturestudy.

Ping Yu et al. present an interesting work involving implementation of a multi-scale dynamic terrain system suitable for application in real-time terrain renderingof local regions [32]. However, the work focuses primarily on the subdivision of theterrain mesh based on an existing deformation algorithm, which is not a focus of thisresearch and a feature already supplied by Unreal Engine 4.23. This paper thereforedoes not provide a strong enough basis for the implementation in this thesis.

7

8 Chapter 3. Method

BTH Summon Search 1Search Term real-time terrain displacementFilters Last 5 years & Subject terms: computer scienceDate searched 2020-03-02Results 108Selected 4 [20, 17, 19, 6]

Figure 3.2: Details of the first BTH Summon database search of the small literaturestudy.

The database search presented in Figure 3.2 yielded four scientific works thatwere examined further.

The 2019 work of Lee and Shin proposes a method of achieving real-time land-scape visualization using hardware-based adaptive terrain meshes through temporalcoherence [20]. While the work covers powerful optimization techniques for real-timeterrain rendering, the main focus remains on the continuous rendering of the terrainmesh rather than the method of terrain deformation. Again, this covers areas nottargeted by the research questions and is not sufficient as a basis for the implemen-tation.

González et al. cover explorations into combining the methods of displacementmapping and parallax mapping presented in Chapter 2 [17]. The work shows promis-ing results for a combination of these methods for improved visual results and reducedtexture stretching and may warrant further exploration in Unreal Engine. However,as with the aforementioned work the target of this lies outside the scope of this thesisand is left for future work.

Lee et al. present an alternative displacement mapping approach with a focus onenhanced detailing [19]. Their work covers vertex relocation as a means of concentrat-ing a fixed amount of vertices in areas with high deformation to achieve high detaillevels without the addition of extra triangles, as a tessellation based approach would.As this thesis however involves the landscape systems of Unreal Engine which relyon a rectilinear grid of vertices, this method would require a redesign of the module.For this reason, this method was not considered for implementation.

Further use cases of displacement mapping methods can be seen in the work ofCondonnier et al., exploring interactive snow simulations [6]. The work presents aninteresting application of the displacement mapping technique. However it focusesmore heavily on fluid dynamics and the behaviour of the simulated snow, and does notcover any specific displacement mapping rendering technique that may be suitablefor implementation.

3.2. Depth-based DDHM 9

BTH Summon Search 2Search Term real-time heightmap displacementFilters Last 5 yearsDate searched 2020-03-04Results 17Selected 2 [22, 27]

Figure 3.3: Details of the second BTH Summon database search of the small litera-ture study.

The work of Nießner et al. covers the usage of hardware tessellation in combi-nation with displacement mapping techniques for real-time rendering [22]. Whiletessellation has the potential to increase the visual quality of the technique, it alsoheavily impacts performance in Unreal Engine 4.23 [16]. This is detailed further inSection 3.3.

Further use case coverage of displacement mapping is provided in the work ofSmelik et al., presenting application through the use of dynamic run-time terraindisplacement in a military simulation environment [27]. While the work does providea good basis for emphasis on the importance of such techniques, it does not coverany technicalities behind the techniques discussed, nor does it cover any performanceaspects. The work is therefore not deemed suitable as a base for implementation.

BTH Summon Search 3Search Term real-time dynamic terrain displacementFilters Subject terms: computer scienceDate searched 2020-03-11Results 227Selected 1 [2]

Figure 3.4: Details of the third BTH Summon database search of the small literaturestudy.

Aquilio et al. present a work that covers a real-time simulation of dynamic terrainusing a Dynamically-Displaced Height Map (DDHM) [2]. The method presented intheir work utilizes simple, core graphics rendering elements that have correspondingcomponents available in most commercial game engines, including Unreal Engine4.23. The proposed deformation technique is shown to achieve real-time performanceas well as present application areas within 3D-games, which covers the criterion forthe research question of this thesis. This work is therefore selected as the primarybasis of the implementation and thus, the small literature study is concluded.

3.2 Depth-based DDHM

The DDHM based terrain displacement method presented by Aquilio et al. utilizes afew core rendering components that are available for use in Unreal Engine 4.23. Themethod involves using a secondary camera to render a depth-only pass of the terrain

10 Chapter 3. Method

from below along with any deforming objects. The depth buffer image generated bythe render pass can then be converted to a heightmap as the elevation values willcorrespond to depth values in the buffer.

Figure 3.5: The depth capture camera configuration for generation of a DDHM [1].

The initial setup of the technique places a secondary camera in the scene directlybelow the terrain. Once a depth render has been made of the landscape to a rendertarget, the render target values need to be mapped to the values of the originalheightmap used to render the terrain. This is done through the use of a conversionscalar, resulting in a linear transformation from the render target depth buffer spaceto heightmap space. After this has been established, further depth buffer renders ofthe terrain can be made with the inclusion of deforming objects. Any objects seenpassing through the terrain will be rendering to the depth map render target by thecamera. Then, once the render target depth buffer is converted into a heightmapthrough the use of the conversion scalar, the terrain will be deformed in the locationswhere the objects pass through the terrain. This process is visualised in Figure 3.6.

3.3. Visual Representation 11

(a)(b)

Figure 3.6: The process of depth information being converted to a heightmap throughthe DDHM technique. (a) shows the object depth clipping through the terrain andbeing rendered to the depth capture render target. (b) shows the depth value databeing used to repopulate the height map. [1]

3.3 Visual Representation

(a) (b) (c)

Figure 3.7: Example of one iteration of the terrain deformation. (a) shows theoriginal unmodified terrain. (b) shows the sphere mesh used for this deformationexample, rendered for demonstration purposes. (c) shows the resulting deformationin the terrain.

Figure 3.7 shows an example of one iteration of deformation performed on a typicallandscape in a game level. Note the minor errors in the heightmap in Figure 3.7c asa result of colour banding due to resolution discrepancy in the depth buffer. This isfurther shown in Figure 3.8 after multiple iterations since each subsequent iterationhas the potential to introduce further errors.

This resolution inaccuracy can potentially be mitigated through the use of hard-ware tessellation in combination with a higher resolution render target, as covered

12 Chapter 3. Method

in Chapter 2. The tessellation options currently available in Unreal Engine 4.23 butintroduce a rather large performance overhead [16]. Due to this, the implementationdescribed in the following chapter will not include any usage of hardware tessellation,as it is not vital to the proposed research questions. Potential approaches to includethis is discussed further in Chapter 8.

Figure 3.8: Example of height levels converging to set stages as a result of colourbanding in the depth buffer. This example shows the result on the landscape after30 iterations.

Chapter 4

Implementation

This chapter describes the implementational details of the method presented in Sec-tion 3.2.

4.1 Deformation Actor

The deformation actor is the object in the scene that controls the rendering of thedepth-map as well as the updating of the height-map data in the landscape actor.The actor contains several components necessary for one iteration of the deformationalgorithm.

4.1.1 Actor Initialization

Before the first iteration of the deformation algorithm, the actor needs to be ini-tialized as demonstrated in Algorithm 2. The render target is created using thedimensions given by the BaseLandscape input actor. Memory for the necessary ar-rays is also reserved at this point. The original pre-deformation heightmap is savedas the heightmap may need to be reset in the future. At this point, the upper andlower value bounds of the heightmap are recorded. The DepthCaptureCamera ac-tor is then set up in a position directly below the origin of the landscape, rotatedpointing upwards and rotated 180 degrees in the Z-axis as the original heightmap isgenerated as seen from above. A capture of the scene from the perspective of theDepthCaptureCamera is made to the render target and the upper and lower boundsof the values in the render target are recorded. The conversion scalar between thetwo ranges is then calculated by dividing the resolution of the original heightmap bythe resolution of the resulting render target values.

4.1.2 Class Structure

The class structure of the deformation actor contains a few core components. Theprimary camera capture actor utilized for capturing the depth image for DDHMgeneration is provided through the USceneCaptureComponent2D class. The actoralso contains references to ALandscape and an instance of UTextureRenderTarget2Dused to access the heightmap and the depth buffer render target. The actor alsocontains functions for each of the algorithms presented in this section, as well asfunctions to call these from an Unreal Engine-blueprint in order to access them atrun-time. The full source code for the deformation actor is detailed in Appendix A.

13

14 Chapter 4. Implementation

(a) (b)

Figure 4.1: One iteration of the implementation performed on a flat landscape,visualized in gray, deforming around a cube shaped mesh, visualized in orange. (a)shows the configuration before the deformation. (b) shows the result of one iterationof the implementation.

4.2. Engine Source Code Modifications 15

Algorithm 1: Importing the heightmap from the Render Target and con-verting to a readable heightmap.input : BaseLandscape, DepthCaptureCamera, HeightmapLowerBound,

RenderTargetLowerBound, ConversionScalar,RenderTargetResource

output: DeformedHeightValues/* Read the depth values from the Render Target */

1 OutputRTHeightmap← RenderTargetResource.ReadFloat16Pixels();2 foreach Height value h in OutputRTHeightmap do

/* Convert the range in the Render Target to the range of theHeightmap */

3 NewHeight = ((h−RenderTargetLowerBound) ∗ConversionScalar) +HeightmapLowerBound;

4 ImportedHeightV alues.Add(NewHeight);5 end/* As the values are captured from below, the heightmap is

mirrored. We need to flip the values in the array */6 width←RenderTargetResource.GetSizeX();7 height←RenderTargetResource.GetSizeY();8 foreach row x in ImportedHeight do9 foreach column y in ImportedHeight do

10 DeformedHeightValues.Add(ImportedHeightValues[width− 1− x+(height− 1− y)× width]);

11 end12 end

4.2 Engine Source Code Modifications

Along with the implementation, a few modifications to the source code of UnrealEngine 4.23 had to be made to make use of a subset of functions during run-time aswell as for testing purposes. These changes are confined to the landscape module andinvolve the removal of definition blocks containing WITH_EDITOR that limit thecompilation of some functions to the editor-only build of the engine. Some necessaryindividual functions such as GetLandscapeInfo in the Landscape.cpp file also containrun-time checks for the variable GIsEditor. These checks have also been removed inthe implementation.

These landscape tools are intended by the developer to be used by the engine withthe landscape editing functionality in the Unreal Engine editor. However, the code iscapable of run-time execution and will execute without error as long as the definitionblocks are removed to provide full initialization of the code at run-time [8]. Table4.1 lists all occurrences of editor checks that were removed in the implementation.While as mentioned, the module is usable at run-time through these modifications, itmay lack some optimizations that are otherwise found in code intended for run-timeuse. It is also worth noting that while no fatal errors were encountering during this

16 Chapter 4. Implementation

Algorithm 2: Initialization of the deformation actor.input : BaseLandscape, DepthCaptureCamera, DepthCameraDistanceoutput: RenderTargetLowerBound, HeightmapLowerBound,

ConversionScalar, RenderTargetResource1 Origin, LandscapeBounds← get extents of BaseLandscape;2 create RenderTarget;3 RenderTargetResource← RenderTarget.GameThread_GetRenderTargetResource();

4 reserve memory for heightmap arrays;5 BaseHeightmap← original unmodified heightmap data;6 foreach vertex v in BaseHeightmap do7 if v > HeightmapHigherBound then8 HeightmapHigherBound← v;9 else if v < HeightmapLowerBound then

10 HeightmapLowerBound← v;11 end

/* DepthCaptureCamera setup */12 set DepthCaptureCamera projection mode to Orthographic;13 set RenderTarget as DepthCaptureCamera texture target;14 set DepthCaptureCamera render mode to ShowOnlyList;

/* Position the DepthCaptureCamera */15 DepthCaptureCamera.ShowOnlyActors.Add(BaseLandscape);16 DepthCaptureCamera.Position ← (Origin.x,Origin.y,

DepthCameraDistance);17 DepthCaptureCamera.Rotation ← (0, 90, 180);

/* Render the scene to RenderTarget using theDepthCaptureCamera */

18 DepthCaptureCamera.CaptureScene();19 HeightData← ReadFloat16Pixels(RenderTarget); foreach vertex v in

HeightData do20 if v > RenderTargetHigherBound then21 RenderTargetHigherBound← v;22 else if v < RenderTargetLowerBound then23 RenderTargetLowerBound ← v;24 end25 RenderTargetResolution← RenderTargetHigherBound ÷

RenderTargetLowerBound;26 HeightmapResolution← HeightmapHigherBound ÷

HeightmapLowerBound;27 ConversionScalar ← HeightmapResolution÷RenderTargetResolution;

4.3. Engine Limitations 17

Algorithm 3: Update the landscape heightmap using values generated fromAlgorithm 1.input: DeformedHeightValues, BaseLandscape/* Create an instance of FHeightmapAccessor */

1 HeightmapAccessor ← FHeightmapAccessor(BaseLandscape);2 HeightmapAccessor.SetData(DeformedHeightValues);3 clean up heightmap arrays;

implementation, run-time usage is not documented by Epic Games and may causeissues when utilized in methods not covered by this thesis.

The UpdateHeightfieldRegion function in UpdateHeightfieldRegion function con-tains one line referencing editor only functionality. The height field reference variableRBHeightfieldEd was changed to instead utilize RBHeightfield which is available atrun-time.

All occurrences of editor specific definition blocks and checks such as#if WITH_EDITOR and check(GIsEditor); were removed from the following files.Filepath Occurrences

Runtime/Landscape/Classes/Landscape.h 7Runtime/Landscape/Classes/LandscapeInfo.h 1Runtime/Landscape/Classes/LandscapeProxy.h 5Runtime/Landscape/Private/Landscape.cpp 5Runtime/Landscape/Private/LandscapeCollission.cpp 1Runtime/Landscape/Private/LandscapeEdit.cpp 4Runtime/Landscape/Private/LandscapeEditInterface.cpp 1Runtime/Landscape/Public/LandscapeEdit.h 2

Table 4.1: All modified engine source code files.

The LandscapeEdit.h and LandscapeEditInterface.cpp files were also edited fortesting purposes. In LandscapeEdit.h, the SetData function on line 348 was editedto always consider the bUpdateFoliage boolean to be false, as this research focuseson simply modifying the heightmap. Similarly in LandscapeEditInterface.cpp, theSetHeightData() function on line 220 was edited to always consider the InUpdate-Collision boolean to be false for a subset of the test cases. These aforementionedtest cases are marked as “No Collision Update” in Section 5.

4.3 Engine Limitations

This section details various limitations of Unreal Engine 4.23 that impacted theefficacy of the method or altered the implementation.

18 Chapter 4. Implementation

4.3.1 Limitations of the USceneCaptureComponent2D

The engine class USceneCaptureComponent2D is used to capture and generate therender target depth map that is later converted into a heightmap for the terraindeformation. As the precision of the depth buffer is determined by the distance be-tween the near and the far plane, demonstrated in Equation 4.1, which is derivedfrom the orthographic projection matrix presented in Figure 4.2, it would be benefi-cial to limit these planes to enclose the scene as close-fitting as possible for optimalprecision. However, as of Unreal Engine version 4.23, it is currently not possibleto set a far plane distance in a USceneCaptureComponent2D manually. This limitsthe options for depth buffer precision to what the engine internally determines as anoptimal far plane given the scene. We can manipulate this by varying the distanceof the USceneCaptureComponent2D from the scene; this is covered further in theResults section.

A simple solution to this problem would be to allow direct accessing of the viewmatrix used in the USceneCaptureComponent2D. This would enable the developerto freely change and define the variables used for the view transformation, such asthe distance of the near and far planes.

A further precision affecting limitation of the USceneCaptureComponent2D isthe 16-bit depth of the depth buffer used for the render target. While the internalheight map for the landscape is stored in 16-bit format, as mentioned, we do nothave explicit control over the depth range due to the inability to access the far planeof the camera. Depth precision is therefore lost once it exceeds the areas covered bythe height values in the landscape. This has the potential to be further impactedby floating-point precision inaccuracies in the depth buffer, particularly at distancesclose to the near plane [25, 23].

4.3.2 Limitations of the Landscape Module

2

right− left0 0 0

02

top− bottom0 0

0 0−2

(far − near)0

−right+ left

right− left− top+ bottom

top− bottom− far + near

(far − near)1

Figure 4.2: The orthographic projection matrix. The direction parameters left, right,top, bottom represent the size of the orthographic projection from the center. Theparameters far and near represent the distance from the camera of the far and nearplanes respectively.

The landscape module API used to access and alter the heightmap of the land-scape mesh through the use of the FHeightmapAccessor interface does not provideexplicit access to the device resource used by the landscape. This results in the im-plementation needing to fetch the resource from the device every iteration insteadof performing operations directly on device data. This adds significant overhead tothe method, and omission of this step would significantly improve the performance

4.3. Engine Limitations 19

because fewer data communications are needed between the CPU and the graphicsdevice. The cost of this operation is covered in Section 5.1.3.

z′ = 2× z − near

far − near− 1 (4.1)

Chapter 5

Results

The experimental results in terms of heightmap accuracy and performance are pre-sented in this chapter. All of the following test cases include recalculation of thenormals on the landscape mesh as per the default implementation in the landscapemodule. A subset of the test cases excludes the updating of the collision mesh. Thesecases are marked with “No Collision Update”.

Each test case consists of a single Unreal Level, containing a single Landscapeactor as well as the DeformableLandscape actor described in the previous section.Each function test is timed in C++ through the high resolution time stamps, andeach individual test is executed through the editor calling the function in the De-formableLandscape actor.

The tests are ran on three different sizes of landscape. Each landscape size con-tains a set number of landscape components, with the smallest size representing asingle landscape component consisting of the default size of 255 by 255 vertices. Eachfurther larger landscape represents an square increase in size from the center of thelandscape, consisting of 3 by 3 and 5 by 5 landscape components respectively. Thesesizes were chosen to give an accurate representation of a simple square level layout ofvarious sizes. As all of the operations are performed per mesh and vertex, the actualsize and scaling of the components in unreal units is irrelevant, and performance willtherefore remain the same regardless.

5.1 Experimental Results

This section covers the testing platform and the quantitative data gathered on theimplementation.

5.1.1 Testing Environment

All of the data presented was gathered from testing the implementation on a com-puter running Microsoft Windows 10 Pro Version 10.0.18363 Build 18363. The test-ing environment is comprised of a Intel Core i7-4770k at 3.5GHz, a Radeon Fury Xwith 4GB of video memory using video driver version 19.9.2, and 16GB of RAM at1600Mhz. All measurements were taken using the FPlatformTime Unreal Engine in-terface, which internally uses the highest resolution timer available on the platform.For the testing platform, this utilizes the QueryHighPerformanceCounter() Win32API function [11]. All tests were performed in a non-debug, developmental build of

21

22 Chapter 5. Results

the engine. The developmental build configuration includes a majority of the opti-mizations applied to a full release build, while still including statistic and profilingtools otherwise not available in the “Shipping” release build configuration. [15]

5.1.2 Full Algorithm Performance

The execution times for one iteration of the full algorithm are presented in Table5.1 and Table 5.2 for the cases involving collision updating and no collision updatingrespectively. A further breakdown of the data for full algorithm execution in the caseof single component with collision updating disabled is shown in Figure 5.1.

2468101214161820222426

Read RenderTarget Set Heightmap Data Full Function

Execution

time(m

s)

No Collision Update, Single Component, Execution Time

Figure 5.1: Execution time of the algorithms performed on a single landscape com-ponent with no collision updating.

Execution time of the full functionLandscape Size Average Max Min1017x1017 144.9ms 160.4ms 109.5ms763x763 80.37ms 116.9ms 58.86msSingle Component(255x255) 22.21ms 25.31ms 8.002ms

Table 5.1: The results of the execution time of a full iteration with collision updatingenabled.

5.1. Experimental Results 23

Execution time of the full function - No CollisionLandscape Size Average Max Min1017x1017 141.2ms 200.7ms 132.1ms763x763 73.10ms 119.2ms 38.32msSingle Component(255x255) 20.48ms 25.62ms 16.44ms

Table 5.2: The results of the execution time of a full iteration with collision updatingdisabled.

5.1.3 Render Target Import Performance

The execution times for the render target import detailed in Algorithm 1 are pre-sented in Table 5.3. As this involves a graphics device command and not any callsto the landscape API, the results are the same whether collision updating is enabledor not, therefore only one table is presented. It is also worth noting that the land-scape sizes listed corresponds to the sizes of the render targets in accordance withthe implementation.

Execution time of the render target importLandscape Size Average Max Min1017x1017 42.33ms 54.67ms 39.72ms763x763 28.97ms 33.60ms 5.765msSingle Component(255x255) 16.94ms 19.32ms 5.044ms

Table 5.3: The results of the execution time of the render target import.

5.1.4 Heightmap Update Performance

Collision Update

Execution time of the heightmap updateLandscape Size Average Max Min1017x1017 103.5ms 124.2ms 96.79ms763x763 42.13ms 61.09ms 38.68msSingle Component(255x255) 7.871ms 11.58ms 7.000ms

Table 5.4: The results of the execution time of Algorithm 3 with collision updatingenabled.

24 Chapter 5. Results

Non-collision Update

Execution time of the heightmap update - No CollisionLandscape Size Average Max Min1017x1017 67.18ms 129.07ms 56.78ms763x763 35.30ms 60.59ms 31.64msSingle Component(255x255) 3.38ms 4.29ms 3.22ms

Table 5.5: The results of the execution time of Algorithm 3 with collision updatingdisabled.

5.1.5 Average Error

Multiple iterations of the method on the landscape mesh can introduce an errordue to insufficient resolution in the depth buffer, as shown in Figure 3.8. The datapresented in Figure 5.2 shows the progression of the average error per heightmapvertex compared to the original heightmap over the course of multiple iterations.The depth capture camera is placed at various distances in the Z-axis away from theorigin of the landscape actor.

0

100

200

300

400

500

600

1 2 3 4 5 6 7 8 9 10AverageError

perVertex(U

nrealU

nits)

Iterations

010005000

1000020000400005000080000

Figure 5.2: Average error distance per vertex with the DepthCaptureCamera placedwith the near plane at various distances from the landscape mesh.

5.1.6 Conversion Resolution Scalar

Due to the resolution discrepancy between the depth buffer in the DepthCapture-Camera in comparison to the original resolution of the heightmap array as describedin Section 4.3.1, a conversion scalar is needed to convert depth values to convertthese values into usable height values for the DDHM technique. Table 5.6 showsthe resolution discrepancy of the depth buffer heightmap at various distances of theDepthCaptureCamera near plane from the landscape actor.

5.1. Experimental Results 25

Resolution Conversion Scalar - Heightmap Resolution 2910Distance from Near Plane Render Target Resolution Scalar0 1576 1.8461000 1050 2.7715000 537 5.41910000 281 10.3620000 140 20.7940000 70 41.5750000 70 41.5780000 32 90.94

Table 5.6: Depth buffer heightmap resolution and resolution conversion scalar atvarious distances of the DepthCaptureCamera.

Chapter 6

Analysis and Discussion

This chapters covers the results presented in Chapter 5 and provides a basis forconclusions of the stated research questions.

6.1 DDHM

The DDHM run-time deformation technique was shown to be fully implementablein the engine in Chapter 4.

Regarding performance, the technique achieved run-time performant deformationon single landscape components, displaying an average frame time of 20.48 millisec-onds, enough to fulfill the criterion of 33.33 milliseconds required for a frame rate of30 frames per second. The performance of the reading of the render target, however,sets a bottleneck for the smaller landscape sizes accounting for roughly 83% of theaverage frame time in the single component case. This is in contrast to the largerlandscape sizes where the heightmap update takes up the majority of the executiontime, see Section 5.1.2.

The technique suffers from accuracy problems as shown in Figure 5.2. Due tothe inability to explicitly set a far plane for the USceneCaptureComponent2D usedin the implementation, the technique cannot fully utilize the Z-range of the depthbuffer. Further precision issues may be attributed to floating point inaccuracies asthe bit depth of the depth buffer is limited to 16-bits, as covered in Section 4.3.This may also be a contributing factor to the seemingly high inaccuracies observedat distance 0 from the near plane. Due to this, more explicit control of the depthcapture camera, as well as the depth buffer, is needed to fully utilize this method,and reduce the resulting error. The error can be mitigated by adjusting the distanceof the component from the actors involved, however, it cannot be fully omitted.

6.2 Generic Run-time Deformation

The execution times for a generic technique-irrelevant heightmap update are pre-sented in Table 5.4 and Table 5.5 respectively. This data shows promising resultsfor dynamic run-time heightmap updates of individual landscape components. Theexecution times exceeding 33.33ms for the larger, full-landscape updates rule out thefeasibility of large scale deformation, however individual components can be con-structed to provide local deformations in areas where the player may be active. Ascomponents are handled as individual meshes in the engine, the possibility exists

27

28 Chapter 6. Analysis and Discussion

for threading and streaming of the updates, which may provide feasible larger scaledeformation.

6.3 SustainabilityAs of the writing of this thesis, the most recent publicly released version of theUnreal Engine game engine is version 4.25, two revisions ahead of the 4.23 versionused in the implementation. These revisions, however, contain no major changes tothe landscape module that would affect the implementation in any way.

Further developments of the engine and the underlying graphics APIs may im-prove the performance of the implementation. As discussed in the previous sections,an optimization of render target read times will significantly improve performancein the case of the individual landscape component, as the majority of the executiontime is spent on the render target read.

6.4 Validity ThreatsOne possibly validity threat could be that the specific version of the engine usedcontains parameters that affect the results in some way. As stated together with theformation of the research questions, this validity threat has been attempted to bemitigated through the analysis of newer versions of the engine. It was concluded thatthe landscape modules do not contain any significant changes in the newest versionof the engine, however, as the tests have not been ran on this version the possibilitycannot be fully dismissed.

While the testing platform was shown to be equivalent to an average user platformat the time of this thesis, it also cannot be fully excluded the possibility that thisspecific platform used has specifications that would in some way directly affect thevalidity of the results.

Chapter 7Conclusions

This thesis aimed to investigate the viability of achieving run-time terrain deforma-tion in the Unreal Engine 4.23 game engine, as well as the feasibility of implementingsaid technique at a performant level, defined as being able to maintain a frame rateof 30 frames per second. The following research questions were posed:

1. Does a run-time terrain deformation technique for arbitrary mesh deformationexist that can be implemented in Unreal Engine 4.23?

2. How well can we achieve run-time performant terrain deformation in UnrealEngine 4.23?

The identified Dynamically-Displaced Height Map technique presented in Chapter3 was shown to be fully implementable and feature-complete in Unreal Engine 4.23,answering research question 1. However, the implementation suffers from accuracyproblems due to limitations of the game engine resulting in limited precision of thedepth buffer. Details regarding the inaccuracy are presented in Section 5.1.5.

Research question 2 touches on the practical feasibility of run-time performantterrain deformation in the engine in general. Rum-time performant was defined inSection 1.2 as being able to achieve a steady frame rate of at least 30 frames persecond. The evaluated Dynamically-Displaced Height Map technique was shownto achieve run-time performant execution times sustainable for a frame rate of 30frames per second when executed on single landscape components. In this test case,the execution time of the technique-specific render target import was shown to bethe primary limiting factor, with the generic case operations constituting a fractionof the frame time. Run-time performant terrain deformation is therefore concludedto be achievable in Unreal Engine 4.23 for individual landscape components, how-ever caution should be exercised when performing deformations on large, full-scalelandscapes.

29

Chapter 8Future Work

For future work, there are many potential routes to further explore this area. Thiswork covers the exploration of the viability of dynamic run-time terrain in UnrealEngine 4.23 using one identified technique, however, there are likely many morewhich may be more suitable for application in a game.

As the specific implementation is limited by certain factors related to the engineitself, covered partially in Section 4.3, future versions of Unreal Engine may enablefor further improvements in some areas. These may include things such as an im-provement in depth map resolution by having access to alter the near plane distanceof the depth capture camera.

A further way to improve the implementation would be to look at a solutionthat performs all operations directly on the render target, which would eliminatethe overhead of a queued render target read operation. This would significantlyimprove the performance of the algorithm as the primary execution time cost forminor deformations in the terrain is the reading of the render target, as seen inChapter 5.

As mentioned in Chapter 2, the technique can be expanded upon through the useof tessellation with a higher resolution height map and depth buffer. This providesfurther detail and higher precision of more finely detailed deformations at the costof the performance overhead of including the tesselation stage in the pipeline.

With the introduction of Turing mesh shaders to the DirectX 12 API, moreopportunities for expansion of the technique are possible [18]. As mesh shaders candecompose large meshes into meshlets, the landscape can be handled as a large entityrather than the separate components as in the current implementation, which mayprovide further performance improvements. The mesh shader pipeline is also vastlydifferent to the more traditional pipeline and will most likely require alterations tothe implementation presented in this thesis.

31

References

[1] Anthony S. Aquilio, Jeremy C. Brooks, Zhu Ying, and G. Scott Owen. Real-time gpu-based simulation of dynamic terrain. In Lecture Notes in ComputerScience (LNCS), volume 4291, page 894, 2006.

[2] Anthony S. Aquilio, Jeremy C. Brooks, Zhu Ying, and G. Scott Owen. Real-time gpu-based simulation of dynamic terrain. In Lecture Notes in ComputerScience (LNCS), volume 4291, pages 891–900, 2006.

[3] Brian Ashcraft. How cover shaped gaming’s last decade. Accessed at: https://kotaku.com/how-cover-shaped-gamings-last-decade-5452654, Jan 2010.

[4] X. Chen and Y. Zhu. Shader based polygon stitching and its application indeformable terrain simulation. In 2011 Sixth International Conference on Imageand Graphics, pages 885–890, Aug 2011.

[5] Mark Claypool, Kajal Claypool, and Feissal Damaa. The effects of frame rateand resolution on users playing first person shooter games. Proceedings of SPIE- The International Society for Optical Engineering, 6071, 01 2006.

[6] G. Cordonnier, P. Ecormier, E. Galin, J. Gain, B. Benes, and M. -. Cani. In-teractive generation of time-evolving, snow-covered landscapes with avalanches.Computer Graphics Forum, 37(2):497–509, 2018.

[7] Valve Corporation. Steam hardware & software survey: April2020. Accessed at: https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam, May 2020.

[8] Dark-Veil. 26495-terrain editing in runtime. Accessed at: https://forums.unrealengine.com/unreal-engine/feedback-for-epic/26495-terrain-editing-in-runtime?p=295744#post295744, Apr 2015.

[9] EricChadwick, Noors, Cheeseplus, and Caustic. Parallax map. Accessed at:http://wiki.polycount.com/wiki/Parallax_Map, Feb 2017.

[10] Alexander Frisk. Real-time vehicle trails in deformable terrain. Master’s thesis,Umeå University, Department of Computing Science, 2017.

[11] Epic Games. Fplatformtime. Accessed at: https://docs.unrealengine.com/en-US/API/Runtime/Core/Unix/FPlatformTime/index.html.

[12] Epic Games. Landscape materials. Accessed at: https://docs.unrealengine.com/en-US/Engine/Landscape/Materials/index.html.

33

34 References

[13] Epic Games. Landscape technical guide. Accessed at: https://docs.unrealengine.com/en-US/Engine/Landscape/TechnicalGuide/index.html.

[14] Epic Games. Unreal units. Accessed at: https://docs.unrealengine.com/udk/Three/UnrealUnits.html, 2012.

[15] Epic Games. Build configurations reference. Accessed at:https://docs.unrealengine.com/en-US/Programming/Development/BuildConfigurations/index.html, May 2020.

[16] Epic Games. Performance guidelines for artists and designers. Ac-cessed at: https://docs.unrealengine.com/en-US/Engine/Performance/Guidelines/index.html, May 2020.

[17] Cesar González, Mariano Pérez, and Juan M. Orduña. Combining displacementmapping methods on the gpu for real-time terrain visualization. The Journal ofSupercomputing, 73(1):402–413, 2017.

[18] Christoph Kubisch. Introduction to turing mesh shaders. Accessed at: https://devblogs.nvidia.com/introduction-turing-mesh-shaders/, Sep 2017.

[19] Eun-Seok Lee, Jin-Hee Lee, and Byeong-Seok Shin. Vertex relocation: a feature-preserved terrain rendering method for pervasive computing environments. Mul-timedia Tools and Applications, 75(22):14057–14073, 2016.

[20] Eun-Seok Lee and Byeong-Seok Shin. Hardware-based adaptive terrain meshusing temporal coherence for real-time landscape visualization. Sustainability,11(7):2137, 2019.

[21] Jamie Madigan. The psychology of video game immersion. Accessedat: https://www.psychologytoday.com/us/blog/mind-games/201207/the-psychology-video-game-immersion, Jul 2012.

[22] M. Nießner, B. Keinert, M. Fisher, M. Stamminger, C. Loop, and H. Schäfer.Real-time rendering techniques with hardware tessellation. Computer GraphicsForum, 35(1):113–137, 2016.

[23] Outerra. Floating point depth buffer. Accessed at: https://outerra.blogspot.com/2009/12/floating-point-depth-buffer.html, Dec 2009.

[24] Jesper Persson. Volume-preserving deformation of terrain in real-time. Master’sthesis, Linköping University, Information Coding, 2019.

[25] Nathan Reed. Depth precision visualized. Accessed at: https://developer.nvidia.com/content/depth-precision-visualized, Jul 2015.

[26] M. Satran, M. Jacobs, and D. Batchelor. Direct3d 11 features - win32apps. Accessed at: https://docs.microsoft.com/en-us/windows/win32/direct3d11/direct3d-11-features?redirectedfrom=MSDN.

References 35

[27] Ruben Smelik, Freek van Wermeskerken, Robbert Krijnen, and Frido Kuijper.Dynamic synthetic environments: a survey. The Journal of Defense Modelingand Simulation, 16(3):255–271, 2019.

[28] Bethesda Softworks. Bethesda.net. Accessed at: http://www.bethsoft.com/.

[29] Kenneth Styrberg. Displacement maps. Accessed at: https://download.blender.org/documentation/htmlI/ch11s03.html, Aug 2005.

[30] Alex Vlachos, Jörg Peters, Chas Boyd, and Jason L. Mitchell. Curved pn tri-angles. In Proceedings of the 2001 Symposium on Interactive 3D Graphics, I3D’01, page 159–166, New York, NY, USA, 2001. Association for Computing Ma-chinery.

[31] Ewan Wilson. A history of environmental destruction. Accessed at: https://thegamesedge.com/602/a-history-of-environmental-destruction/#.g7DzjgrMBa, Sep 2018.

[32] Ping Yu, Tao Xu, Bo Zheng, and Yanfeng Zhang. Research and implementationof multiscale dynamic terrain. In 2018 International Conference on Security,Pattern Analysis, and Cybernetics (SPAC), pages 217–222. IEEE, 2018.

Appendix ADeformation Actor Source Code

DeformableTerrainActor.h

#pragma once

#inc lude "RomeMinimal . h"#inc lude "GameFramework/Actor . h"#inc lude "Components/SceneCaptureComponent2D . h"#inc lude "Landscape . h"#inc lude "LandscapeEdit . h"#inc lude "LandscapeProxy . h"#inc lude "LandscapeInfo . h"#inc lude " LandscapeEditorUt i l s . h"#inc lude "Engine/TextureRenderTarget2D . h"#inc lude " . . / Fo l i age / Fo l i a g eC l a s s e s . h"//#inc lude " Ins tancedFo l iageActor . h"#inc lude "DeformableTerrainActor . generated . h"

UCLASS( )c l a s s ROME_API ADeformableTerrainActor : pub l i c AActor{

GENERATED_BODY()

pub l i c :// Sets d e f au l t va lue s f o r t h i s actor ’ s p r op e r t i e sADeformableTerrainActor ( ) ;

UFUNCTION( BlueprintPure ,Category = "Deformable Terra in " ,DisplayName = "Get Deformable Terra in Actor " ,

meta = (WorldContext = "WorldContextObject " ,UnsafeDuringActorConstruct ion = " true ") )

ADeformableTerrainActor∗ Get ( ) ;

UFUNCTION( Bluepr in tCa l l ab l e , Ca l l InEd i to r )void Ed i t o r I n i t ( ) ;

UFUNCTION( B luep r in tCa l l ab l e )

37

38 Appendix A. Deformation Actor Source Code

void ImportHeightFromRenderTarget ( ) ;

UFUNCTION( B luep r in tCa l l ab l e )void ResetHeightmapToDefault ( ) ;

p ro tec ted :UPROPERTY( VisibleAnywhere , Category = "Base S e t t i n g s ")USceneCaptureComponent2D∗ DepthCaptureCamera ;

UPROPERTY(EditAnywhere , Category = "Base S e t t i n g s ")ALandscape∗ BaseLandscape ;

UPROPERTY(EditAnywhere , Category = "Base S e t t i n g s ")UTextureRenderTarget2D∗ HeightmapRenderTarget ;

// Cal led when the game s t a r t s or when spawnedv i r t u a l void BeginPlay ( ) ov e r r i d e ;

v i r t u a l void EndPlay (const EEndPlayReason : : Type EndPlayReason ) ov e r r i d e ;

p r i va t e :TArray<uint16> Orig ina lHeightData ;in t32 MinX, MinY, MaxX, MaxY;FTextureRenderTargetResource∗ RenderTargetResource ;FIntRect SampleRect ;TArray<uint16> HeightData ;TArray<FFloat16Color> OutputRTHeightmap ;

f l o a t s c a l e r = 1 ;in t32 RTDepthLowerBound = 0 ;in t32 HeightmapLowerBound = MAX_int32 ;

void ResetHeightMap ( ) ;void I n i t ( ) ;long double Ca l cu la teEr ro r (TArray<uint16>& newArray ) ;

pub l i c :// Cal led every framev i r t u a l void Tick ( f l o a t DeltaTime ) ove r r i d e ;

} ;

39

DeformableTerrainActor.cpp

#inc lude "DeformableTerrainActor . h"#inc lude "Mate r i a l s /Mater ia lInstanceDynamic . h"#inc lude "LandscapeStreamingProxy . h"// Only f o r t e s t i n g / l ogg ing purposes#inc lude <iostream>#inc lude <fstream>

// Sets d e f au l t va lue sADeformableTerrainActor : : ADeformableTerrainActor ( ){

PrimaryActorTick . bCanEverTick = f a l s e ;DepthCaptureCamera =CreateDefaultSubobject<USceneCaptureComponent2D>(TEXT("Depth Capture Camera " ) ) ;RootComponent = DepthCaptureCamera ;HeightmapRenderTarget =NewObject<UTextureRenderTarget2D >() ;

}

ADeformableTerrainActor∗ ADeformableTerrainActor : : Get ( ){

re turn t h i s ;}

void ADeformableTerrainActor : : Ed i t o r I n i t ( ){

I n i t ( ) ;}

void ADeformableTerrainActor : : ImportHeightFromRenderTarget ( ){

u int64 startTime ;u int64 endTime ;

ALandscape∗ Landscape = BaseLandscape ;ULandscapeInfo∗ LandscapeInfo =Landscape−>GetLandscapeInfo ( ) ;HeightData . Empty ( ) ;

startTime = FPlatformTime : : Cycles64 ( ) ;RenderTargetResource−>ReadFloat16Pixe ls (

OutputRTHeightmap , CubeFace_MAX) ;f o r ( FFloat16Color Color : OutputRTHeightmap ){

uint16 Height =( uint16 ) ( ( Color .R. Encoded − RTDepthLowerBound)

40 Appendix A. Deformation Actor Source Code

∗ s c a l e r ) + HeightmapLowerBound ;HeightData .Add( Height ) ;

}

TArray<uint16> FlippedHeightData ;FlippedHeightData . Reserve ( HeightData .Num( ) ) ;u int16 ∗ data = HeightData . GetData ( ) ;f o r ( i n t x = 0 , width = RenderTargetResource−>GetSizeX ( ) ;x < width ; ++x){

f o r ( i n t y = 0 ,he ight = RenderTargetResource−>GetSizeY ( ) ;y < he ight ; ++y){

FlippedHeightData .Add( data [ width − 1 − x +( he ight − 1 − y ) ∗ width ] ) ;

}}

FHeightmapAccessor<f a l s e > HeightmapAccessor (LandscapeInfo ) ;

HeightmapAccessor . SetData (MinX, MinY, MaxX, MaxY,FlippedHeightData . GetData ( ) ) ;long double e r r o r = Ca lcu la t eErro r ( FlippedHeightData ) ;

// Clean up ar raysHeightData . Empty ( ) ;

Fl ippedHeightData . Empty ( ) ;OutputRTHeightmap .Empty ( ) ;

}

void ADeformableTerrainActor : : ResetHeightmapToDefault ( ){

ResetHeightMap ( ) ;}

// Cal led when the game s t a r t s or when spawnedvoid ADeformableTerrainActor : : BeginPlay ( ){

Super : : BeginPlay ( ) ;

I n i t ( ) ;}

void ADeformableTerrainActor : : EndPlay (const EEndPlayReason : : Type EndPlayReason ){

41

Super : : EndPlay ( EndPlayReason ) ;}

void ADeformableTerrainActor : : ResetHeightMap ( ){

ULandscapeInfo∗ LandscapeInfo =BaseLandscape−>GetLandscapeInfo ( ) ;FHeightmapAccessor<f a l s e > HeightmapAccessor (LandscapeInfo ) ;TArray<uint16> Orig ina lHeightData2 ;HeightmapAccessor . SetData (MinX, MinY, MaxX, MaxY,Orig ina lHeightData . GetData ( ) ) ;

}

void ADeformableTerrainActor : : I n i t ( ){

ULandscapeInfo : : RecreateLandscapeInfo (GetWorld ( ) , f a l s e ) ;ULandscapeInfo∗ LandscapeInfo =BaseLandscape−>GetLandscapeInfo ( ) ;LandscapeInfo−>GetLandscapeExtent (MinX, MinY, MaxX, MaxY) ;

RenderTargetResource = HeightmapRenderTarget−>GameThread_GetRenderTargetResource ( ) ;

SampleRect = FIntRect (0 , 0 , FMath : : Min(1 + MaxX − MinX,HeightmapRenderTarget−>SizeX ) ,

FMath : : Min(1 + MaxY − MinY,HeightmapRenderTarget−>SizeY ) ) ;

// Make sure ar rays are c l eanOrig ina lHeightData . Empty ( ) ;HeightData . Empty ( ) ;OutputRTHeightmap .Empty ( ) ;

// Reserve memory f o r heightmap ar raysin t32 HeightmapSize =SampleRect .Width ( ) ∗ SampleRect . Height ( ) ;Or ig ina lHeightData . Reserve ( HeightmapSize ) ;OutputRTHeightmap . Reserve ( HeightmapSize ) ;HeightData . Reserve ( HeightmapSize ) ;

FHeightmapAccessor<f a l s e >HeightmapAccessor ( LandscapeInfo ) ;// Store the o r i g i n a l unmodif ied he ight dataHeightmapAccessor . GetDataFast (MinX, MinY, MaxX, MaxY,Orig ina lHeightData . GetData ( ) ) ;

42 Appendix A. Deformation Actor Source Code

HeightmapLowerBound = MAX_int32 ;in t32 HeightmapHigherBound = MIN_int32 ;f o r ( i n t i = 0 ; i < HeightmapSize ; ++i ){

auto a = Orig ina lHeightData . GetData ( ) [ i ] ;i f ( a > HeightmapHigherBound ){

HeightmapHigherBound = a ;}

i f ( a < HeightmapLowerBound ){

HeightmapLowerBound = a ;}

}

// Place the camera under the t e r r a i nFTransform Modif iedTransform = GetTransform ( ) ;FVector Origin , BoundsExtent ;BaseLandscape−>GetActorBounds (f a l s e , Origin , BoundsExtent ) ;DepthCaptureCamera−>OrthoWidth = BoundsExtent .X ∗ 2 ;SetActorTransform (Modif iedTransform ) ;

RenderTargetResource−>ReadFloat16Pixe ls (OutputRTHeightmap , CubeFace_MAX) ;

in t32 RTHeightmapLowerBound = MAX_int32 ,RTHeightmapHigherBound = MIN_int32 ;f o r ( FFloat16Color Color : OutputRTHeightmap ){

uint16 Height = ( uint16 ) Color .R. Encoded ;HeightData .Add( Height ) ;i f ( ( u int16 ) Color .R. Encoded > RTHeightmapHigherBound ){

RTHeightmapHigherBound = ( uint16 ) Color .R. Encoded ;}i f ( ( u int16 ) Color .R. Encoded < RTHeightmapLowerBound){

RTHeightmapLowerBound = ( uint16 ) Color .R. Encoded ;}

}

in t32 temp = RTHeightmapHigherBound −RTHeightmapLowerBound ;

in t32 temp2 = HeightmapHigherBound − HeightmapLowerBound ;RTDepthLowerBound = RTHeightmapLowerBound ;s c a l e r = ( f l o a t ) temp2 / ( f l o a t ) temp ;

}

43

long double ADeformableTerrainActor : : Ca l cu la t eErro r (TArray<uint16> &newArray ){

long double count = newArray .Num( ) ;u int16 ∗ origData = Orig ina lHeightData . GetData ( ) ;u int16 ∗ newData = newArray . GetData ( ) ;long double averageError = 0 . 0 ;f o r ( i n t i = 0 ; i < count ; ++i ){

averageError += abs ( ( long double ) or igData [ i ] −( long double ) newData [ i ] ) / count ;}

re turn averageError ;}

// Cal led every framevoid ADeformableTerrainActor : : Tick ( f l o a t DeltaTime ){

Super : : Tick ( DeltaTime ) ;}

Faculty of Computing, Blekinge Institute of Technology, 371 79 Karlskrona, Sweden