y Leveraging Ray Tracing Hardware Acceleration In Unity · 2019-05-31 · Leveraging Ray Tracing...
Transcript of y Leveraging Ray Tracing Hardware Acceleration In Unity · 2019-05-31 · Leveraging Ray Tracing...
Ge
ne
rati
ve A
rt —
Ma
de
wit
h U
nit
y
Leveraging Ray Tracing Hardware Acceleration In UnityDigital Dragons ‘19
Anis Benyoub
Graphics Engineer@Auzaiffe
Replace with image
High Definition Render Pipeline (HDRP)
3
Scriptable Render Pipeline
Modern design
Designed for AAA productions
Physically Based Lighting
Book of the dead
https://github.com/Unity-Technologies/ScriptableRenderPipeline
Replace with image
HDRP
4
Harold HalibutThe Heretic
Replace with image
Motivation
5
Ge
ne
rati
ve A
rt —
Ma
de
wit
h U
nit
y
6
Replace with image
Ray Tracing Effects
7
Replace with image
Ray Tracing Effects
8
Left: real footage. Right rendered with Unity
Replace with image
Ray Tracing Effects
9
Ambient Occlusion
Replace with image
Ray Tracing Effects
10
Ambient Occlusion Indirect Diffuse (GI)
Replace with image
Ray Tracing Effects
11
Ambient Occlusion Indirect Diffuse
Recursive Tracing
Replace with image
Ray Tracing Effects
12
Ambient Occlusion Indirect Diffuse
Indirect Specular
(Reflections)
Recursive Tracing
Replace with image
Ray Tracing Effects
13
Ambient Occlusion Indirect Diffuse
Indirect Specular Stochastic Area Shadows
Recursive Tracing
Replace with image
Ray Tracing Effects
14
Ambient Occlusion Indirect Diffuse
Indirect Specular Stochastic Area Shadows
Recursive Tracing
Replace with image
Ray Tracing Effects
15
Ambient Occlusion Indirect Diffuse
Indirect Specular Stochastic Area Shadows
Recursive Tracing
Hybrid (Rasterization/Ray Tracing) Renderer
Replace with image
HDRP Render Graph
16
A wild digital dragon appears!
Replace with image
Render Graph - Deferred/Forward
17
HDRP’s hybrid deferred/forward render graph
Replace with image
Render Graph - Deferred/Forward
18
HDRP’s hybrid deferred/forward render graph
(blue are optional steps)
Replace with image
Render Graph - Deferred/Forward
19
Replace with image
Render Graph - Deferred/Forward
20
Replace with image
Render Graph - Deferred/Forward
21
Replace with image
Render Graph - Deferred/Forward
22
Replace with image
Render Graph - Deferred/Forward
23
Replace with image
Render Graph - Forward Only
24
HDRP’s forward only render graph
Replace with image
Cornell Box
25
Cornell Box in HDRP with stochastic area shadows and ray traced indirect diffuse
Replace with image
Cornell Box
26
Replace with image
Cornell Box
27
Replace with image
Classic Command List
28
Replace with image
Classic Command List
29
SSR, SSAO, Conact Shadows…
Replace with image
Classic Command List
30
Replace with image
Classic Command List
31
Prepass, GBuffer, Forward
Opaque, Forward Transparents, …
Replace with image
Classic Command List
32
Replace with image
Classic Command List
33
SSR, SSAO, Conact Shadows…
Replace with image
DXR API
34
Ray Tracing Acceleration
StructureRay Tracing Shader
Pipeline
Replace with image
Ray Tracing Acceleration Structure
35
Replace with image
Ray Tracing Acceleration Structure
36
BLAS
Replace with image
Hybrid Renderer Command List
37
Replace with image
Hybrid Renderer Command List
38
Replace with image
Hybrid Renderer Command List
39
Replace with image
Hybrid Renderer Command List
40
Replace with image
Missing Features
41
Replace with image
Missing Features
42
Replace with image
Missing Features
43
Replace with image
Missing Features
44
Replace with image
Missing Features
45
Replace with image
Missing Features
46
Replace with image
Missing Features
47
Replace with image
Surface vs Effect Type Shaders
48
Replace with image
Surface vs Effect Type Shaders
49
Replace with image
Frustum Culling vs Volume Culling
50
Replace with image
Frustum Culling vs Volume Culling
51
Replace with image
Camera Filtering vs Per Effect Filtering
52
Replace with image
Camera Filtering vs Per Effect Filtering
53
Replace with image
Tiled Lighting
54
Replace with image
Cluster Lighting
55
Replace with image
Render Graph - Deferred/Forward
56
HDRP’s hybrid deferred/forward render graph
Replace with image
Hybrid Render Graph
57
HDRP’s hybrid rasterization/ray tracing render graph
Replace with image
Hybrid Pipeline
58
● Blue solved using an integrator (path tracer, photon mapper, etc.)
● Needs to be rephrased for our hybrid approach
Replace with image
Hybrid Pipeline
59
● Red evaluated using rasterization (almost)
● Blue evaluated using rasterization or raytracing
Replace with image
Hybrid Pipeline
60
● Indirect specular is approximated as an Isotropic GGX lobe
● Indirect diffuse is approximates as a Lambert lobe
● Ambient occlusion affect indirect diffuse or both
Replace with image
Ray Generation
61
Depth Buffer
Replace with image
Ray Generation
62
Depth Buffer
Replace with image
Ray Generation
63
Depth Buffer
Replace with image
Ray Generation
64
Depth Buffer
Replace with image
Sampling Based Integration
65
● Our budget is 1spp per effect per frame (or less)
● We use spatio-temporal reprojection/accumulation to increase
the coverage
○ Spatial coverage improved by “Blue-noise Dithered
Sampling” [Georgiev and Fajardo 2016]
○ Temporal coverage improved by iterating
over a Sobol sequence
Replace with image
Sampling Based Integration
66
Scrambling Tile 256x256x2
Owen Scrambled Sobol Sequence256x1x4
Replace with image
Indirect Specular
67
Replace with image
Indirect Specular
68
Replace with image
Indirect Specular
69
*
Replace with image
Indirect Specular
70
*
Replace with image
Indirect Specular
71
● Split Sum approximation [Karis 2013] is useful for variance reduction
● Blue is precomputed
● Red evaluated with rasterization or ray tracing
*
Real Shading in Unreal Engine 4 (Brian Karis)
Replace with image
Indirect Specular
72
Small office scene rendered with ray traced indirect specular
Replace with image
Indirect Specular
73
Variable Smoothness
*
Ray traced indirect specular term
Replace with image
Indirect Specular
74
Smoothness = 0.0
Ray traced indirect specular term
Replace with image
Indirect Specular
75
Replace with image
Indirect Specular
76
Replace with image
Indirect Specular
77
Replace with image
Indirect Specular
78
Replace with image
Indirect Specular
79
Replace with image
Indirect Specular
80
38 ms for 1 spp
2080Ti @ 1920x1080
Replace with image
Indirect Specular
81
Replace with image
Indirect Specular
82
Replace with image
Indirect Specular
83
17 ms for 1 spp
38 ms for 1 spp
2080Ti @ 1920x1080
Replace with image
Indirect Specular
84
CS
Replace with image
Indirect Specular
85
CS
Replace with image
Indirect Specular
86
CS
Replace with image
Indirect Specular
87
CS
Replace with image
Indirect Specular
88
CSCS
Replace with image
Indirect Specular
89
8.74ms for 1 spp
17 ms for 1 spp
38 ms for 1 spp
2080Ti @ 1920x1080
Replace with image
Ray Binning
90
”It Just Works”: Ray-Traced Reflections in ’Battlefield V’Johannes Deligiannis Jan Schmid EA DICE
Replace with image
Ray Binning
91
Screen tiles of 32x32 pixels for binning
Replace with image
Ray Binning
92
Screen tiles of 32x32 pixels for binning
Generated rays in the unit sphere for a tile
Replace with image
Ray Binning
93
Octahedral space for ray direction binning
Generated rays in the unit sphere for a tile
Replace with image
Indirect Specular
94
Replace with image
Indirect Specular
95
CS
Replace with image
Indirect Specular
96
CS CS
Replace with image
Indirect Specular
97
CS CS
Replace with image
Indirect Specular
98
CS CS
Replace with image
Indirect Specular
99
CS CS
Replace with image
Indirect Specular
100
CSCS CS
Replace with image
Indirect Specular
101
6.8ms for 1 spp
8.74ms for 1 spp
17 ms for 1 spp
38 ms for 1 spp
2080Ti @ 1920x1080
Replace with image
Indirect Specular
102
Variable Smoothness (Min Smoothness = 0.6)
Replace with image
Indirect Specular
103
2.7 ms for 1 sppVariable Smoothness (Min Smoothness = 0.6)
2080Ti @ 1920x1080
Replace with image
Indirect Specular – Additional compromises
104
”It Just Works”: Ray-Traced Reflections in ’Battlefield V’Johannes Deligiannis Jan Schmid EA DICE
Variable rate tracing
Mixing with ray marching
Replace with image
Area Shadow
105
Replace with image
Area Shadow
106
Replace with image
Area Shadow
107
Sponza Scene - Reference
Replace with image
Area Shadow
108
Replace with image
Area Shadow
109
Replace with image
Area Shadow
110
Variance Reduction?
Replace with image
Area Shadow
111
Sponza Scene - LTC only
“Real-Time Polygonal-Light Shading with Linearly Transformed Cosines”Eric Heitz, Jonathan Dupuy, Stephen Hill and David Neubelt
Replace with image
Area Shadow
112
“Combining Analytic Direct Illumination and Stochastic Shadows”Eric Heitz, Stephen Hill and Morgan McGuire
Sponza Scene - LTC + Stochastic Shadow
Replace with image
Area Shadow - Visibility
113
Replace with image
114
Area Shadow - Stochastic
Replace with image
115
Area Shadow - Stochastic
Replace with image
116
Area Shadow - Stochastic
Replace with image
117
Area Shadow - Stochastic
Replace with image
118
Area Shadow
LTC
Un
Sn
Replace with image
119
Area Shadow - Stochastic
Replace with image
120
Area Shadow - Stochastic
CS CS
Replace with image
121
Area Shadow - Stochastic
12.8 ms for 4 spp
2080Ti @ 1920x1080
Replace with image
122
Area Shadow - Stochastic
CS CS
Replace with image
123
Area Shadow - Stochastic
Use simplestshader variant
CS CS
Replace with image
124
Area Shadow - Stochastic
9.9 ms for 4 spp
12.8 ms for 4 spp
2080Ti @ 1920x1080
Replace with image
125
Area Shadow - Stochastic
CS
Replace with image
126
Area Shadow - Stochastic
CS
Replace with image
127
Area Shadow - Stochastic
CS
Replace with image
128
Area Shadow - Stochastic
CS CS CS
Replace with image
129
Area Shadow - Stochastic
5 ms for 4 spp
9.9 ms for 4 spp
12.8 ms for 4 spp
2080Ti @ 1920x1080
Replace with image
Recursive tracing
130
Replace with image
Alpha Blend Transparency
131
Replace with image
Alpha Blend Transparency
132
Replace with image
Alpha Blend Transparency
133
Replace with image
Alpha Blend Transparency
134
Replace with image
Alpha Blend Transparency
135
Replace with image
Screen Space Refraction
136
Replace with image
Screen Space Refraction
137
??
Replace with image
Screen Space Refraction
138
Replace with image
Ray Traced Transparency
139
Replace with image
Ray Traced Transparency
140
Replace with image
Ray Traced Transparency
141
Replace with image
Ray Traced Transparency
142
Replace with image
Ray Traced Transparency
143
Replace with image
Ray Traced Transparency
144
Replace with image
Hybrid Transparent Render Graph
145
Replace with image
146
Takeaways
● Naive is possible but not viable performance wise
● Reduce variance with analytic approximation/precomputation
● Forward materials are a real problem
● Sometimes rasterization is good enough
Replace with image
147
Takeaways
● Bin rays when distribution requires it
● Avoid anyhit as much as possible
● Take advantage of compute shaders when possible
● Some rasterization paradigms are incompatible with ray tracing
Replace with image
148
Takeaways
● Manage resources lifetime to avoid render target allocation
● Temporal reprojection/accumulation is key for denoising
● For real time, ray budget is extremely tight
● It might not make your image prettier, simply more accurate
Replace with image
149
Future Improvements
● Mix ray traced effects with their rasterized variant
● Lower resolution effects and up-sample all effects
● Use ray tracing for non rendering applications
● Path tracer for reference
● Skinned meshes and particle support
Replace with image
150
Acknowledgement
Unity Graphics TeamSebastien Lagarde (Unity)Kate McFadden (Unity)Dany Ayoub (Unity)Alexandre Cepisul (L&S)Awen Couellan (L&S)Aymeric du Chéné (L&S)Mike Geig (Unity)Emmanuel Turquin (Unity)Laurent Harduin (Unity)Ionut Nedelcu (Unity)Arnaud Carré (Unity)Joel de Vahl (Unity)Tim Cooper (Unity)
Eric Heitz (Unity)Francesco Cifariello Ciardi (Unity)Antoine Lelievre (Unity)Cyril Jover (Unity)Lewis Jordan (Unity)Natasha Tatarchuk (Unity)Jesper Mortensen (Unity)Tian Ning (Unity)Melissa Chou (Unity)
Replace with image
151
Questions
https://github.com/Unity-Technologies/ScriptableRenderPipeline
Replace with imageAdditional materials
152
Replace with image
Indirect Specular
153
Replace with image
Indirect Specular
154
Replace with image
155
Area Shadow
LTC
Un
Sn
Replace with image
156
Area Shadow
LTC
Un
Sn
denoise(Un)
denoise(Sn)
Replace with image
157
Area Shadow
denoise(Un)
denoise(Sn)
denoise(Sn) / denoise(Un)
Replace with image
158
Area Shadow
denoise(Sn) / denoise(Un) LTC LTC + Stochastic Shadow