Adaptive Streaming and Rendering of Large Terrains

Raphaël Lerbour Advisors:

Kadi Bouatouch (IRISA)

Jean-Eudes Marvie (THOMSON)

2

Terrain rendering

2D maps of elevation and color samples

3D applications: games, GPS, virtual tourism…

3

Objectives

Render large terrain datasets

– Support huge maps (dozens of GB)

– Remote loading

With good interactivity

– Unpredicted user viewpoint moves

– Speed requirements (e.g. 25 frames per second)

– Real-time hardware 3D rendering

On multiple target devices and networks

– Desktop PCs, handhelds…

– Cannot transmit or display entire dataset

Solutions

Network limitations: adaptive client-server streaming

– Progressive loading guided by rendering needs

– Available subset of dataset is rendered while loading

Graphics hardware limitations: adaptive rendering

– 3D mesh simplification (data selection)

4

File serverServer

database

NetworkStreamingcontroller

Partialdatabase

Server Client

5

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

6

Related work

Commercial software

– Google Earth, NASA World Wind…

– Flight simulators

Continuous levels of detail (LOD)[Lindstrom et al. 1996, Duchaineau et al. 1997, Hoppe 1998]

– Focus on reducing mesh complexity at all costs

– Every data sample is considered at each frame

– Designed for slow 3D rendering hardware

Alternative approaches– Clipmap [Losasso et al. 2004]

– Projective grid [Dachsbacher et al. 2004]

Google Earth

NASA World Wind

Blocks with discrete levels of detail[De Boer 2000, Pouderoux et al. 2005, Schneider et al. 2006]

– Map subdivided into fixed-size blocks

– Per-block selection between predefined LODs

– Blocks can be loaded independently

– All blocks needed to render entire map

Hierarchies of blocks[Levenberg 2002, Cignoni et al. 2003, Livny et al. 2007]

– All blocks have the same resolution

– Blocks organized in a tree

– Changing LOD requires changing tree level

– Tree can be loaded progressively

Related work

7

Split

Discussion

Related work

– Few solutions support both streaming and rendering efficiently

– Some are based on outdated considerations

– Few support planetary terrains and solve corresponding issues

Our approach

– Unify adaptive data streaming and selectionSame data structure and adaptive schemes, suited for both

– But separate from 3D renderingGeneric streaming and selection for large 2D maps, then application to 3D terrains

– Adapt to any database size and hardware capabilitiesFavor speed, avoid data redundancy, support planetary terrains

– Offer a complete and efficient solution

8

9

Overview of our work

Generic adaptive solution

– Streaming and rendering of sample maps

Application to 3D terrain rendering

– Support of planetary terrains

Offline preprocessing steps with support for huge maps

File server

Serverdatabase

Network Adaptivestreaming

and selection

Partial clientdatabase

3D conversionand rendering

Preprocessing

Originaldatabase

10

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

11

Hierarchy of square blocks [Levenberg et al. 2002]

– Can be progressively loaded as a tree, starting with the root

– Hierarchical block selection minimize amount of rendered blocks

Blocks have sets of regular levels of detail (LOD) [De Boer 2000]

– Adaptive LOD selection minimize amount of structure operations

Data structure: base

12

Data structure: new properties

No data redundancy: less to store and transmit

– LODs of a block share data (common sample grid)

– Parent and children share one LOD (local copy when split/merge)

New LOD: samples interleaved between existing ones

– Possible to render a block with not all LODs loaded

– Possible to render a block and load one of its LODs in parallel

13

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

14

Measure of importance

The base for adaptivity in both streaming and rendering

– Importance represents desired quality for a block

– Thresholds to select LODs and trigger update requests

– Defines priority for triggered requests

Any factors may be used

– Based on application, nature of the data

– Typical factors for 3D terrain rendering:Distance from viewpointSurface areaRoughness

15

Adaptive streaming

Pre-computed server database for minimal activity

– Only one file read per LOD request

– File position for any LOD known from previous LOD

Data are loaded “as-is”

– Conversion on the client, in parallel with rendering

– Possible quantization, pre-compression

– Same server and data with any client type

We always transmit the most important data

– Implicit adaptation to the network speed

– Rendering quality constantly improves

0.1 s (at 1Mbps)

10 s

40 s

16

Adaptive streaming

Restricted number of pending requests

– Potential LOD loading requests come with an importance value

– Requests with highest importance are transmitted to server

– Others are discarded, need to update importance for next selection

– At reception, database is updated and new request is transmitted

Server

File server

NetworkClient

Adaptivestreaming

Adaptiverendering

Renderingsystem

Partialdatabase

Completedatabase

Importance

New LOD

Request

Reply (data)

17

Client database

Incomplete tree of blocks

– Data on all leaves entire surface can be rendered

– Asynchronous update operations

– Tree update requests selected the same way as loading requests

Server

Partialdatabase

Adaptivestreaming

Treeupdate

Adaptiverendering

RefineSplit &merge

Rendering

Requests Requests Adaptiveselection

At each frame, we:

– Hierarchically cull blocks that are not visible

– Compute importance for each block to select LOD to be rendered

– Trigger requests when unavailable LODs are selected

– Select most important requests from triggered ones

– Render visible leaf blocks at selected LOD using sample masks

Adaptive rendering

18

LOD0

19

User selects desired rendering speed

– Adaptive quality factor in importance formula

– Guides selection of LODs and triggering of operations

Adaptive selection

LOD1

Importance

LOD0 LOD-1

LOD1 usedLOD0 requested, LOD1 used

SplitMergeLOD0 usedChildren used

50 frames/s

100 frames/s

150 frames/s

Results

Tested with 3D terrain rendering application

– CPU usage: about 5-10% of each frame time

– No stutters due to database updates (asynchronous)

– Rendering quality improves as fast as network allows

– Stable frame rate as long as enough data are loaded

Scales well from high to low performance clients (set-top box)

20

Puget Sound database(Washington state, USA)

21

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

22

Hardware 3D rendering

We render 3D geometry with triangles

– Conversion from elevation in refine operation

– Adapted measure of importance

– 3D culling with bounding boxes

Geometry caching in graphics hardware

– Well suited for discrete LODs

– Saves memory transfers (up to +40% rendering speed)

Sample masks as triangle strips

– Applied directly in hardware

– Need to solve cracks on block edges

23

Fixing geometry cracks

Additional triangle strip masks on block edges

– Block with higher definition adapts

– No new samples required

– Neighbor knowledge: one per edge

– No adaptation needed when more than one

There are unsolvable cases

– Split and merge constraints

24

Texture maps

Color maps rendered using textures

– 1 LOD = 1 mipmap

– Hardware caching and selection

Distinct but linked geometry and texture trees

– Specific measures of importance

– Single texture coordinates mask for all geometry blocks

– Only one texture per geometry block: split and merge constraints

Data filtering for down-sampling

– Improved quality for low definition LODs– Progressive Texture Map [Marvie03]

25

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

Modeling a planet

Datum to support ellipsoid reference shape Sphere projected onto a bounding cube

– Produces square maps

– Saves most redundant data compared toclassical solution (25% global)

– Avoids visual “stretching” on poles

26

Google Earth (global, cylindrical)

Our solution (cube, gnomonic)

27

Projection adjustment

Base: gnomonic 2D map projection

– Fast reverse projection (normalization)

– 75% less surface on corners than center

New adjusted sampling

– Planet surface instead of plane of projection

– Steps = independent angles of rotation

– Two tangent values computed per sample

– 33% less surface on corners than center

Plane ofprojection:

Crack fixing on edges of the cube

– Faces specifically numbered and oriented

– Implicit and transparent management

Runtime adaptive clipping planes

– Good rendering precision, from any distance

– Culls hidden parts of the planet (behind the horizon)

– No additional comparison

28

Solving specific planet issues

0

1 2

3 4

5

Results

Tested on GeForce 9800 GTX+, all features turned on

– 140 FPS when asking for 2 million triangles

29

Earth databasefrom NASA

30

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

Huge input, huge output

– Memory constraints for loading input files

– Linear writing to output files preferred

First step: re-projection of a planetary map

– Project specific points of output window to find input window

– Recursive output map subdivision

– Load input window when fits in memory and re-project samples

Preprocessing

31

1 of 6 outputgnomonic maps

1 global inputcylindric map

Preprocessing

Second step: generation of a server database

– Direct input window computation

– Top-down subdivision into complete tree of blocks

– Load input for any sub-tree when fits in memory

– Bottom-up construction of block LODs and linear file writing

32

Input map Tree of blocks

Results

Support for input and output maps of arbitrary size

– Projection on a cube: -25% database size

– LOD compression with Zlib: up to -75% database size

33

Puget Sound1.25 GB 740 MB

2mn

SRTM174 GB

TrueMarble41.7 GB

9h

1h30

5h

35mn

14.9 GB

6.8 GB

31 GB

126 GB

34

Plan

Introduction

Related Work

Contributions

– Data Structure

– Adaptive streaming and rendering

– 3D terrain rendering features

– Planetary terrains

– Preprocessing

Conclusion

Conclusion

Generic adaptive solution

– Sample maps with any size and type

– Adapts to network and rendering speeds

– Single data structure, generic methods

– No redundant data, fast handling

Application to 3D terrain rendering

– Optimizations for 3D graphics hardware

– Most rendering artifacts avoided

– Uniformly sampled planet surface

– Improved rendering precision

Offline generation of huge databases

35

Future work

Enhance current work

– Validate the solution on handheld devices

– Use GPU shaders for fast and flexible 3D conversion

– Further improve precision with local coordinate systems

– Find more specific factors for measure of importance

Add new features

– Generation of procedural details

– Fix for popping artifacts

– Advanced LOD compression

– Self-shadowing of the terrain

– Alternate rendering system with projective grid

36

37

Publications

Adaptive Streaming and Rendering of Large Terrains: A Generic Solution. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. In WSCG 2009.

Adaptive Real-time Rendering of Planetary Terrains. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. Accepted, to be presented at WSCG 2010.