Practical implementation of light scattering effects using ... · Render coord texture Detect depth...

83
Practical Implementation of Light Scattering Effects Using Epipolar Sampling and 1D Min/Max Binary Trees Egor Yusov Graphics Software Engineer – Intel Corporation

Transcript of Practical implementation of light scattering effects using ... · Render coord texture Detect depth...

Page 1: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Practical Implementation of Light Scattering Effects Using Epipolar Sampling and 1D Min/Max Binary Trees

Egor Yusov

Graphics Software Engineer – Intel Corporation

Page 2: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Demo

2

Page 3: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• Introduction

• Light Scattering Theory

• Solution to Inscattering Integral

• Epipolar Sampling

• 1D Min/Max Binary Trees

• Implementation

• Performance

Agenda

3

Page 4: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• Light scattering greatly enhances the realism

• It is hard to compute

• Achieving high-performance is challenging

Introduction

4

𝐿𝐼𝑛𝑃𝑡 𝐂, 𝐎, 𝐋 = 𝑒−𝑇 𝐏→𝐂 𝛽 𝐏 𝑝 𝜃

𝐼𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐

𝑆𝑂

𝑆𝐶

𝑉 𝐏 𝑑𝑠

𝑇 𝐀 → 𝐁 = 𝛽 𝐏 𝑑𝑠𝐁

𝐀 𝑝𝑅 𝜃 =

3

16𝜋1 + cos2 𝜃

𝑝𝑀 𝜃 =1

4𝜋

3(1 − 𝑔2)

2(2 + 𝑔2)

1 + 𝑐𝑜𝑠2𝜃

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

Page 5: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• Hoffman & Preetham, 2002 – Alanalytical model for outdoor light scattering

• Sun et al, 2005 – Semi-analytical solution to airlight integral

• Tevs et al, 2008 – Using maximum mipmaps for dynamic height field rendering

• Chen et al, 2011 – Using 1d min-max mipmaps for volumetric shadows

• Engelhardt & Dachsbacher, 2010 – Epipolar Sampling

Previous Work

5

Page 6: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Aerial Perspective

For homogeneous medium

6

𝐿𝐶𝑎𝑚 = 𝐿𝑂𝑏𝑗 ∙ 𝑒−𝑇 𝐎→𝐂 +𝐿𝐼𝑛

𝑇 𝐀 → 𝐁 = 𝛽 𝐏 𝑑𝑠𝐁

𝐀

= 𝛽 𝐀 − 𝐁

Camera

Light source

Object

Page 7: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Radiant Intensity of a Point Light

7

• I – radiant intensity

• 𝑣 = 𝐎−𝐂𝐎−𝐂

– view direction

I

𝐿𝐼𝑛 = 𝐼

𝑣

Page 8: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Attenuation and optical thickness

8

• Amount of energy reaching P is

− Inversely proportional to distance squared

− Attenuated by the extinction in the media

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐

P

Page 9: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Visibility factor

9

• 𝑉 𝐏 = 1 if point is lit and 0 if shadowed

− Filtered in practice

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐 𝑉 𝐏

P

Page 10: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Scattering coefficient

10

• Total scattered light (in any direction) is proportional to

− Section length 𝑑𝑠

− Scattering coefficient 𝛽 𝐏

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝛽 𝐏 𝑑𝑠

P

𝑑𝑠

Page 11: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Phase function

11

• Phase function gives amount of light scattered towards the camera

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝑝 𝜃 𝛽 𝐏 𝑑𝑠

P 𝑑𝐿𝐼𝑛

𝑑𝐿𝐼𝑛 𝜃

Page 12: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Attenuation of inscattered light

12

• Differential inscattered light is attenuated in the media

• Total inscattering is given by integrating differential amounts

𝐿𝐼𝑛 = 𝐼 𝑒−𝑇 𝐋→𝐏

𝐋 − 𝐏 𝟐 𝑉 𝐏 𝑝 𝜃 𝛽 𝐏 𝑒−𝑇 𝐏→𝐂 𝑑𝑠

𝐎

𝐂

P 𝑑𝐿𝐼𝑛

𝜃

𝑒−𝑇 𝐏→𝐂

Page 13: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Inscattering Integral Derivation

Directional light

• Phase function does not vary across the ray

• Light intensity is constant

13

𝐿𝐼𝑛 = 𝑝 𝜃 𝑒−𝑇 𝐏→𝐂 𝛽 𝐏 𝑉 𝐏 𝐸𝑆𝑢𝑛

𝐎

𝐂

𝑑𝑠

Page 14: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Scattering Properties of the Media

Rayleigh scattering

• Caused by molecules

• Wavelength-dependent

𝛽𝑅 . rgb = (5.8, 13.5, 33.1) × 10−6

• Almost isotropic:

𝑝𝑅 𝜃 =3

16𝜋(1 + cos2 𝜃)

14

Page 15: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Scattering Properties of the Media

Mie Scattering

• Caused by aerosols

• Wavelength-independent

𝛽𝑀. rgb = 2.0 × 10−5

• Anisotropic

𝑝𝑀 𝜃 =1

4𝜋

3(1 − 𝑔2)

2(2 + 𝑔2)

1 + 𝑐𝑜𝑠2𝜃

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

15

Page 16: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Scattering Properties of the Media

Scattering due to both types of particles

• 𝑇 𝐀 → 𝐁 = 𝛽Σ 𝐀 − 𝐁

𝛽Σ = 𝛽𝑅 + 𝛽𝑀

• 𝛽𝑝 𝜃 → 𝛽𝑅 ∙ 𝑝𝑅 𝜃 + 𝛽𝑀 ∙ 𝑝𝑀 𝜃 ≡ 𝑃Σ (𝜃)

16

Page 17: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Scattering Properties of the Media Isotropic phase

function

17

𝑝𝑀 𝜃 =1

4𝜋

Page 18: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Scattering Properties of the Media Anisotropic phase

function

18

𝑝𝑀 𝜃 = 1

4𝜋

3(1 − 𝑔2)

2(2 + 𝑔2)∙

1 + 𝑐𝑜𝑠2𝜃

1 + 𝑔2 − 2𝑔𝑐𝑜𝑠𝜃 3/2

Page 19: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Solution to Inscattering Integral

Directional light – fully analytical solution

𝐿𝐼𝑛𝐷𝑖𝑟 𝜃, 𝑆 =

𝐸𝑆𝑢𝑛 ∙ 𝑃Σ (𝜃)

𝛽Σ1 − 𝑒−𝛽Σ∙𝑆

𝑆 = 𝐎 − 𝐂

19

𝜃

𝑆

Page 20: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Solution to Inscattering Integral

𝑠 ℎ

P

𝜃 𝐋0

Point light source – integral reformulation

𝐿𝐼𝑛𝑃𝑡 = 𝐿𝐼𝑛

𝑃𝑡 ℎ, 𝑆𝐶 , 𝑆𝑂 =

𝐼 𝑃Σ (𝜃) 𝑆𝑂

𝑆𝐶

∙𝑒−𝛽Σ 𝑠−𝑆𝐶 + ℎ2+𝑠2

ℎ2 + 𝑠2𝑑𝑠

20

𝐋 − 𝐏 2

𝐏 − 𝐂 + 𝐋 − 𝐏

cos 𝜃 = −𝑠

ℎ2 + 𝑠2

𝑆𝑂 𝑆𝐶

Page 21: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

Solution to Inscattering Integral

𝐋0

Point light source – look-up table

Precompute 2D LUT:

ℒ ℎ, 𝑆 = 𝐿𝐼𝑛𝑃𝑡 ℎ, 𝑆, +∞

21

𝑆

Page 22: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

C

L

O

Solution to Inscattering Integral

𝐋0

Point light source – semi-analytical solution

𝐿𝐼𝑛𝑃𝑡 ℎ, 𝑆𝐶 , 𝑆𝑂 =

ℒ ℎ, 𝑆𝐶

−𝑒−𝛽Σ 𝑆𝑂−𝑆𝐶 ℒ ℎ, 𝑆𝑂

22

𝑇 𝐂 → 𝐎 𝑆𝑂 𝑆𝐶

Page 23: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Volumetric Shadows

23

𝑆0𝐸𝑛𝑡 𝑆0

𝐸𝑥𝑡 𝑆1𝐸𝑛𝑡 𝑆2

𝐸𝑛𝑡 𝑆2𝐸𝑥𝑡 𝑆1

𝐸𝑥𝑡

𝑒−𝛽Σ∙ 𝑆𝑖𝐸𝑛𝑡−𝑆𝐶 ℒ ℎ, 𝑆𝑖

𝐸𝑛𝑡

−𝑒−𝛽Σ∙ 𝑆𝑖𝐸𝑥𝑡−𝑆𝐶 ℒ ℎ, 𝑆𝑖

𝐸𝑥𝑡

𝐿𝐼𝑛𝑃𝑡 =

𝑛−1

𝑖=0

𝑇 𝑆𝑖𝐸𝑥𝑡 → 𝐂

𝑇 𝑆𝑖𝐸𝑛𝑡 → 𝐂

Page 24: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

𝐿𝐼𝑛𝐷𝑖𝑟 = 𝐹(𝜃) ∙ 𝑒−𝛽Σ∙𝑆𝑖

𝐸𝑛𝑡− 𝑒−𝛽Σ∙𝑆𝑖

𝐸𝑥𝑡

𝑛−1

𝑖=0

𝐹 𝜃 =𝐸𝑆𝑢𝑛 ∙ 𝑃Σ (𝜃)

𝛽Σ

Volumetric Shadows

24

𝑆0𝐸𝑛𝑡 𝑆0

𝐸𝑥𝑡 𝑆1𝐸𝑛𝑡 𝑆2

𝐸𝑛𝑡 𝑆2𝐸𝑥𝑡 𝑆1

𝐸𝑥𝑡

Page 25: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Volumetric Shadows

Starting point - tracing the view ray in shadow map space

For each pixel:

•Project the view ray onto the shadow map

•Set up 𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 = ℒ ℎ, 𝑆𝐶 , 𝐿𝐼𝑛. 𝑟𝑔𝑏 = 0

•Step through each texel:

−𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏 = 𝑒−𝛽Σ∙ 𝑆−𝑆𝐶 ℒ ℎ, 𝑆

−𝐿𝐼𝑛. 𝑟𝑔𝑏 += 𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 − 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏 ∙ 𝑉(𝐏)

−𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 = 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏

25

𝑃𝑟𝑒𝑣𝐿𝐼𝑛. 𝑟𝑔𝑏 𝐶𝑢𝑟𝑟𝐿𝐼𝑛. 𝑟𝑔𝑏

Page 26: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Volumetric Shadows

Perspective-correct interpolation

1

𝑧=1

𝑧𝐶+ 𝛼1

𝑧𝑂−1

𝑧𝐶

𝑆

𝑧=𝑆𝐶𝑧𝐶+ 𝛼𝑆𝑂𝑧𝑂−𝑆𝐶𝑧𝐶

26

Shadow

Map plane

𝑆𝐶

𝑆𝑂

𝑆

𝑧𝐶

𝑧𝑂

𝑧 𝛼

1 − 𝛼

Page 27: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Volumetric Shadows

Starting point - tracing the view ray in shadow map space

• Need to test all samples along the ray

• Too slow (> 90 ms on GeForce 680 GTX)

• Optimizations are necessary!

27

Page 28: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

28

Page 29: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

29

Page 30: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

30

Page 31: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Sample generation

31

Exit point

Entry point

Entry point

Exit point

Page 32: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Sample generation – improved sample placement

32

Too dense sampling

Much better sampling

Page 33: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Coordinate texture

33

Epip

ola

r lin

es

Sample coordinate on each line

Page 34: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Sample classification

34

Initial ray marching

samples are placed

equidistantly

Additional ray marching

samples are placed at

depth breaks The remaining samples are interpolation

samples

Page 35: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Interpolation source texture

35

0 1 2 3 4 5 6 7 8

0,0 0,3 0,3 3,3 4,4 4,8 4,8 4,8 8,8

Page 36: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Transformation from epipolar to rectangular coordinates

36

• Cast epipolar line through the pixel

• Perfrom bilateral filtering

− One Gather() and two Sample() instructions are used

Page 37: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

Fix-up pass

37

• Not all samples can be properly interpolated from epipolar coordinates

• These samples are marked in stencil

• Additional fix-up pass is performed without 1D min/max acceleration

Page 38: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Epipolar Sampling

38

91 ms -> 14 ms

We can do better!

Page 39: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

1D min/max binary trees

Depth

1D height map & first level

39

Page 40: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

1D min/max binary trees

Depth

Second level

40

Page 41: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

1D min/max binary trees

Depth

Third level

41

Page 42: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

1D min/max binary trees

Depth

Traversal optimization

42

𝑑𝑚𝑖𝑛

𝐀 𝐁

𝑑𝑚𝑎𝑥

max 𝑑𝐀, 𝑑𝐁

min 𝑑𝐂, 𝑑𝐃

𝐂 𝐃

𝐄 𝐅

• max 𝑑𝐀, 𝑑𝐁 < 𝑑𝑚𝑖𝑛: lit

• min 𝑑𝐂, 𝑑𝐃 > 𝑑𝑚𝑎𝑥: shadowed

• EF: neither lit nor shadowed

Page 43: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Shadow map

Slice Origin and Direction

43

𝑢

𝐎𝑈𝑉 𝐷𝑈𝑉

𝑣

• Location of the i-th sample is

𝐎𝑈𝑉 + 𝑖 ∙ 𝐷𝑈𝑉

𝐎𝑈𝑉 𝐷𝑈𝑉

Page 44: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Slice Origin and Direction

44

𝑆 1

𝑆 0

Exit point View

ray

𝑁𝐿𝑖𝑔ℎ𝑡

𝑁𝑆𝑙𝑖𝑐𝑒

𝑁𝑆𝑙𝑖𝑐𝑒 = 𝑆 1 × 𝑆 0

𝐷 = 𝑁𝑆𝑙𝑖𝑐𝑒 × 𝑁𝐿𝑖𝑔ℎ𝑡

𝐷

𝑆 0 =𝐋 − 𝐂

𝐋 − 𝐂

𝑆 1 =𝐒𝐄𝐱𝐢𝐭 − 𝐂

𝐒𝐄𝐱𝐢𝐭 − 𝐂

𝑂 𝑆ℎ𝑎𝑑𝑜𝑤 𝑚𝑎𝑝 𝑝𝑙𝑎𝑛𝑒

Page 45: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Slice Origin and Direction

45

𝑂 𝑆ℎ𝑎𝑑𝑜𝑤 𝑚𝑎𝑝 𝑝𝑙𝑎𝑛𝑒

Page 46: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Slice Origin and Direction

46

𝑆 1

𝑆 0

𝑁𝐿𝑖𝑔ℎ𝑡

𝑁𝑆𝑙𝑖𝑐𝑒

𝐷

𝑂

𝐏0 = 𝑐1𝑁𝑆𝑙𝑖𝑐𝑒 + 𝑐2𝑁𝐿𝑖𝑔ℎ𝑡 + 𝑡𝐷

𝐏0

𝐎 = 𝐏0 + 𝑡𝑚𝑖𝑛 ∙ 𝑫

𝑡𝑚𝑖𝑛 𝑡𝑚𝑎𝑥

𝑆ℎ𝑎𝑑𝑜𝑤 𝑚𝑎𝑝 𝑝𝑙𝑎𝑛𝑒

Page 47: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Slice Origin and Direction

47

𝑆 1

𝐷

𝑂

𝑁𝑆𝑙𝑖𝑐𝑒

−𝐿

𝐿

𝑆 1

𝐷 𝑂

𝑁𝑆𝑙𝑖𝑐𝑒

−𝐿

𝐿

Page 48: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Colored Light Shafts

48

Page 49: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Auxiliary textures

49

Name Format Dimension

Back Buffer RGBA_UNORM8 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Camera Space Z R_FLOAT32 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Camera Depth Stencil D24_S8 𝑊𝑆𝑐𝑟𝑒𝑒𝑛 × 𝐻𝑆𝑐𝑟𝑒𝑒𝑛

Coordinate texture RG_FLOAT32 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Epipolar depth-stencil D24_S8 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Epipolar camera space Z R_FLOAT32 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Interpolation source RG_UINT16 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Inscattering × 2 RGBA_FLOAT16 𝑁𝑆𝑎𝑚𝑝𝑙𝑒𝑠 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Slice End Points RGBA_FLOAT32 𝑁𝑆𝑙𝑖𝑐𝑒𝑠 ×1

Slice UV origin & direction RGBA_FLOAT32 𝑁𝑆𝑙𝑖𝑐𝑒𝑠 ×1

1D min/max shadow map × 2 RG_FLOAT16 𝐷𝑆ℎ𝑎𝑑𝑜𝑤𝑀𝑎𝑝 × 𝑁𝑆𝑙𝑖𝑐𝑒𝑠

Algorithm specific data

Page 50: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation

1. Reconstruct camera space Z

1. Reconstruct camera space Z

Camera space Z

Camera depth buffer

Camera depth buffer

3. Render coordinate texture

3. Render coordinate texture

Epipolar depth stencil

Epipolar depth stencil

Interpolation

source texture

Interpolation

source texture

5. Render Slice Origin and Direction in

Shadow Map Space

5. Render Slice Origin and Direction in

Shadow Map Space

2. Render Slice End Points in Screen Space

2. Render Slice End Points in Screen Space

Slice End Points

Slice UV Orig & Dir 4. Refine Sample

Locations 4. Refine Sample

Locations

Epipolar camera space Z

Epipolar coordinates

Page 51: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation

Epipolar coordinates

Epipolar depth stencil

Epipolar depth stencil

Interpolation source texture

Interpolation source texture

Camera space Z

7. Mark Ray Marching Samples

7. Mark Ray Marching Samples

8. Do Ray Marching

8. Do Ray Marching

Initial inscattering

51

6. Build 1D min/max mipmap

6. Build 1D min/max mipmap

Shadow map

1D min-max mipmap

Slice UV Orig & Dir

Page 52: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Epipolar

camera space Z

Camera space Z Initial inscattering

9. Interpolate inscattering

9. Interpolate inscattering

Interpolation source texture

Interpolation source texture

Interpolated inscattering

10. Transform to rectangular coordinates

10. Transform to rectangular coordinates

52

Source Color Buffer

Slice End Points

Page 53: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation

Initial inscattering

Source Color Buffer

12. Fix inscattering

12. Fix inscattering

Final image

53

Camera space Z

Shadow map

Page 54: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Discontinuities search with CS

• Each ray section is processed by one thread group

• Each thread processes one sample

54

Thread

Group

Thread

Group

Page 55: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Discontinuities search with CS

• Step 1: each thread sets 1-bit flag in group shared array indicating if there is a depth break

−InterlockedOr() must be used

• Step 2: find closest depth break or section end for each sample using bit manipulation and firstbitlow() and firstbithigh() intrinsics

55

0 0 0 1 0 0 0 0

Up to 6x speed-up

Page 56: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Min/max binary tree construction

56

Page 57: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Min/max binary tree construction

• Step 1: construct 1st tree level using shadow map

− Use Gather() instructions to read min/max z for each pair of two adjacent samples

− Compute conservative min/max z

• Step 2: Construct coarser tree levels

− Use temporary texture to render to

57

𝑢

𝑂

𝐷

𝑣

One texel

Page 58: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Implementation Details

Optimized sample placement

58

-1

+1 2/H

2/W

-1 +1

Less efficient sample

placement

More efficient sample

placement

Gives more than 5% speedup

Page 59: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Performance

59

0

0.5

1

1.5

2

2.5

3

Reconstruct

cam Z

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

Ray march Interpolate Transform to

rect coords

Inscattering fix-

up

ms

2560 × 1600 resolution, NVIDIA GeForce 680 GTX, spot light

Profile # slices # samples SM Time, ms Mem, MB

Brute Force 40962 91.18

High Quality 2048 1024 40962 5.986 88.0

Balanced 1024 1024 20482 3.269 44.0

High performance 1024 512 10242 2.267 18.0

Page 60: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Brute Force Ray Marching

60

Page 61: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Quality

61

Page 62: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Balanced

62

Page 63: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Performance

63

Page 64: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Performance

64

0

5

10

15

Reconstruct

cam Z

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

Ray march Interpolate Transform to

rect coords

Inscattering fix-

up

ms

1366 × 768 resolution, Nvidia Quadro 1000M, spot light

Profile # slices # samples SM Time, ms Mem, MB

Brute Force 20482 164

High Quality 1024 1024 20482 18.9 44.0

Balanced 512 512 10242 8.37 11.0

High performance 512 256 5122 5.38 4.5

Page 65: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Brute Force Ray Marching

65

Page 66: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Quality

66

Page 67: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Balanced

67

Page 68: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Performance

68

Page 69: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Performance

69

0

5

10

15

Reconstruct

cam Z

Render coord

texture

Detect depth

breaks

Build 1D

min/max trees

Ray march Interpolate Transform to

rect coords

Inscattering fix-

up

ms

1024 × 768, Intel CPU with 3rd Gen HD Graphics, spot light

Profile # slices # samples SM Time, ms Mem, MB

Brute Force 20482 184.79

High Quality 1024 1024 20482 28.83 44.0

Balanced 512 512 10242 12.85 11.0

High performance 512 256 5122 8.85 4.5

Page 70: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Brute Force Ray Marching

70

Page 71: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Quality

71

Page 72: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Balanced

72

Page 73: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

High Performance

73

Page 74: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

0.07 0.01 0.09 0.31

0.01

0.38

1.67

0.06

0.52 0.26

0

0.5

1

1.5

2

Reconstruct

cam Z

Render slice

enter/exit

points

Render

coord

texture

Detect depth

breaks

Render slice

origin and

dir

Build 1D

min/max

trees

Ray march Interpolate Transform to

rect coords

Inscattering

fix-up

Time, ms

Performance

74

• Spot light, balanced quality

• 1280 × 960 resolution, NVIDIA GeForce 480 GTX

• 1024 slices, 512 samples in slice, 2048 × 2048 Shadow Map

3.366 ms

Page 75: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

0.07 0.01 0.10 0.31

0.01

0.65

2.12

0.06

0.57

0.20

0

0.5

1

1.5

2

2.5

Reconstruct

cam Z

Render slice

enter/exit

points

Render

coord

texture

Detect depth

breaks

Render slice

origin and

dir

Build 1D

min/max

trees

Ray march Interpolate Transform to

rect coords

Inscattering

fix-up

Time, ms

Performance

75

• Directional light, high quality

• 1280 × 960 resolution, NVIDIA GeForce 480 GTX

• 1024 slices, 512 samples in slice, 4096 × 4096 Shadow Map

4.076 ms

Page 76: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Ray marching, 5.49

Ray marching, 1.67

0.000

1.000

2.000

3.000

4.000

5.000

6.000

No binary trees With 1D min/max trees

Time, ms

Binary tree construction,

0.38 ms

Performance

76

> 2x faster

Page 77: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Performance

Colored light shafts

77

• Binary tree optimization is less efficient

−Ray marching: 1.67 ms -> 5.49 ms

−Total time: 3.36 ms -> 6.86 ms

• Still less than 7 ms on NVIDIA GeForce 480 GTX

Page 78: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• Semi-analytical solution is used to calculate the integral on each ray segment

− No sophisticated mathematics like SVD etc.

• Epipolar scattering is combined with 1D min/max binary trees

− Significant performance improvements

• Efficient implementation methods

Advantages Over Previous Approaches

78

Page 80: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• Large outdoor environments

• Integration with cascaded SM, VSM/EVSM

• Heterogeneous (dynamic) media

− Clouds

− Smoke

− Sparse octree/PRT

Extensions

80

Page 81: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

• HOFFMAN N., PREETHAM A. J.: Rendering outdoor light scattering in real time. In Proceedings Game Developer Conference 2002.

• ENGELHARDT T., DACHSBACHER C.: Epipolar sampling for shadows and crepuscular rays in participating media with single scattering. In Proc. 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, 119–125.

• CHEN J., BARAN I., DURAND F., JAROSZ W.: Real-time volumetric shadows using 1d min-max mipmaps. In Proceedings of the Symposium on Interactive 3D Graphics and Games, 39–46.

• GAUTRON P., MARVIE J.-E., FRANCOIS G.: Volumetric shadow mapping. ACM SIGGRAPH 2009 Sketches.

• TEVS A., IHRKE I., SEIDEL H.-P.: Maximum mipmaps for fast, accurate, and scalable dynamic height field rendering. In Symposium on Interactive 3D Graphics and Games (i3D’08), 183–190.

• SUN B., RAMAMOORTHI R., NARASIMHAN S. G., NAYAR S. K.: A practical analytic single scattering model for real time rendering. ACM Trans. Graph. 24, 3, 1040–1049.

References

81

Page 82: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

The presenter would like to thank:

• Artem Brizitsky

• Jeffery A. Williams

• Doug Mcnabb

• Dave Bookout

• Charu Chandrasekaran

• Quentin Froemke

• Glen Lewis

• Kyle Weicht

Acknowledgments

82

Page 83: Practical implementation of light scattering effects using ... · Render coord texture Detect depth breaks Build 1D min/max trees Ray march Interpolate Transform to rect coords Inscattering

Thank you!

83