Post on 24-Dec-2021
Global Illumination Shadow Layers
François Desrichard, David Vanderhaeghe, Mathias Paulin
IRIT, Université de Toulouse, CNRS, INPT, UPS, UT1, UT2J, France
July 12, 2019
Cinematic Lighting Design
Arbitrary Output Variables (AOVs) allow editing without re-rendering
We provide shadow layers for compositing with little overhead
2
Diffuse, specular, normal, depth...
Animated geometry Textures, materials Light rig
Scene Composite Render
Shadows
Strong visual cue, subject to artistic expression
Tolerant perception of shadow appearance (Hecher et al., 2014, Sattler et al., 2005)
3 Nightmare by Alla Chernova Horizon-Based Ambient Occlusion (HBAO) in Destiny 2
Previous Work
4
Dragging with mouse (Pellacini et al., 2002)
On-surface deformation (Ritschel et al., 2010)
Rotation, pattern inlay (Obert et al., 2010) Shape simplification (DeCoro et al., 2007)
● Light overestimation ● Empirical matte ● No indirect shadows
Comparison with Available Renderers
5
Main layer = original image Arnold / pbrt / Cycles, no shadow Ours, no shadow
Ours, shadow layer Arnold, shadow matte
Comparison with Current Renderers
6
Main layer
Arnold, shadow matte
Ours, shadow layer
Compositing with the shadow layer
Contributions
● Definition of shadow layer under global illumination
● Characterization of the shadow layer in the path space
● Path tracer rendering all layers in a single pass
7
Defining Shadow
Interaction between three components
The user defines caster and catcher by their surface
How to account for indirect shadows? 8
Light
Caster
Catcher
Occlusion can be direct or indirect
Screen Space Definition
9
Black body render B Invisible render T Main layer I
As the difference between the two altered renders: S = T - B
Screen Space Definition
10
- =
Invisible render T Black body render B Shadow layer S
Adding the shadow layer removes shadows
Screen Space Definition
11
= +
Shadow layer with matte No shadow Main layer
Discussion on this Definition
● Agnostic to the rendering algorithm ● Takes into account indirect shadows ● Has a physical meaning: lost radiance
● Self-shadowing cannot be recovered ● No control on light or catcher ● Two additional renders per object
12
Path integral formulation for the main layer (Veach, 1997)
For the shadow layer of caster C
Translation to the Path Space
13
= { all geometric light paths in the scene }
= { light paths encountering caster C }
= measurement contribution function considering C invisible
● Input: N objects as casters Ci
● Output: the main layer and N shadow layers in a single pass
● The algorithm measures with , but also each with
○ Measuring with ⇔ scattering on all casters
○ Measuring with ⇔ skipping caster Ci
Integration in a Path Tracer
14
i i
i } p = 1 / 2
1. Intersect green
Integration in a Path Tracer
15
3. Contribute to Sgreen
4. Cannot skip blue
2. Skip green now
? Sblue
Sgreen
1. Contribute to Sgreen
Integration in a Path Tracer
16
3. Scatter on blue now
4. Contribute to I
2. Intersect blue
?
Improved Artistic Control
Per-light separation
17
Main layer Separate light sources; shadow ratio I / (I + S)
Improved Artistic Control
Direct and indirect separation
18
Main layer Direct and indirect shadow ratio
Improved Artistic Control
Custom catchers and self-shadowing toggle
19
Main layer With and without self-shadows
Editing Examples
Balancing shadow strength
20
Editing Examples
21
+
Hibiscus / Pandanus shadow layer
Locally color graded image
Main layer
Editing Examples
Shape transformation using the shadow ratio I / (I + S)
22
Main layer I (I + S) ⨯ the transformed shadow ratio
Performance Overhead
● Additional intersection tests bring the most overhead
● Managing samples and filtering incurs a per-caster cost
● The sampling budget is now shared among all layers
For 1 to 5 casters, we measure a 1.1 to 1.3 overhead factor
23
Limitations
When a shadow is created by multiple casters, it cannot be assigned to only one of them. A possible solution: consider them as one.
With N interacting casters, combinatorial explosion (2N shadow layers) 24
Main layer Boxes as separate casters Both boxes as one caster
Limitations
Using the notion of surface to define an object is limiting: how to handle participating media?
25 Main layer No scattering No absorption
Conclusion
● Definition of shadow layer under global illumination ○ Editable at the post-processing stage
● Characterization of the shadow layer in the path space ○ Amenable to Monte-Carlo integration algorithms
● Path tracer rendering all layers in a single pass
○ With an overhead in time and render convergence
26
Thank you for your attention
https://www.irit.fr/STORM/site/shadow-layers
https://github.com/frcsdes/shadow-layers
27
References
(DeCoro et al., 2007) DECORO C., COLE F., FINKELSTEIN A., RUSINKIEWICZ S.: Stylized shadows. In Proc. 5th International Symposium on Non-photorealistic Animation and Rendering (New York, NY, USA, 2007), NPAR ’07, ACM, pp. 77–83. doi:10.1145/1274871.1274884.
(Hecher et al., 2014) HECHER M., BERNHARD M., MATTAUSCH O., SCHERZER D., WIMMER M.: A comparative perceptual study of soft-shadow algorithms. ACM Trans. Appl. Percept. 11, 2 (July 2014). doi:10.1145/2620029.
(Obert et al., 2010) OBERT J., PELLACINI F., PATTANAIK S.: Visibility editing for all-frequency shadow design. Computer Graphics Forum 29, 4 (2010), 1441–1449. doi:10.1111/j.1467-8659.2010.01741.x.
(Pellacini et al., 2002) PELLACINI F., TOLE P., GREENBERG D. P.: A user interface for interactive cinematic shadow design. ACM TOG 21, 3 (July 2002), 563–566. doi:10.1145/566654.566617.
(Ritschel et al., 2010) RITSCHEL T., THORMÄHLEN T., DACHSBACHER C., KAUTZ J., SEIDEL H.-P.: Interactive on-surface signal deformation. ACM TOG 29, 4 (July 2010), 36:1–36:8. doi:10.1145/1778765.1778773.
(Sattler et al., 2005) SATTLER M., SARLETTE R., MÜCKEN T., KLEIN R.: Exploitation of human shadow perception for fast shadow rendering. In Proc. 2nd symposium on Applied perception in graphics and visualization (2005), APGV05, Association for Computing Machinery, p. 131–134. doi:10.1145/1080402.1080426.
(Veach, 1997) VEACH E.: Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University, Stanford, CA, USA, 1997. AAI9837162.
28
Usually exported using a light path expression: .* <[RT].’tallbox’> .*
-
Scattered Layer
29
=
Scattered layer Black body render B Main layer I
Scattered Layer
30
Participating Media
31 Shadow layer test Shadow removal test
Integration in a Path Tracer
32
1. Propagation
The first time a caster C is encountered, the path now belongs to
● Measure with ⇒ skip the surface and contribute to S ○ C is assigned to the path and always skipped ○ No other caster may be skipped
● Measure with ⇒ scatter normally and contribute to I ○ C will never be skipped ○ Other casters can still be assigned
Both outcomes have probability p = 1 / 2 and introduce a factor 1 / p
2. Direct lighting
If the path was assigned a caster the shadow ray can skip it, and contributes radiance to the corresponding shadow layer
Otherwise: ● An unoccluded shadow ray contributes to I ● An occluded shadow ray can skip a caster that was never
encountered and contribute to its shadow layer
Overall, less zero radiance paths
Integration in a Path Tracer
33
Future work: locally adapt p to the probability of an encounter?
Skip Probability
34
p = 0 p = 1
Global illumination Direct shadows
Direct illumination Global shadows
p = 1 / 2
Our choice
Scene N Equal sampling Equal time Equal variance
SPP Time SSIM ZRP Time SPP μ-Var Var Time μ-SPP
Teaser
0 2048 15’ 27” 0.903 26% 15’ 2048 0.003 0.01 18’ 24” 2321
1 2048 17’ 27” 0.901 17% 15’ 1824 0.004 0.01 20’ 08” 2318
2 2048 18’ 07” 0.896 12% 15’ 1728 0.004 0.01 21’ 46” 2360
3 2048 18’ 28” 0.896 10% 15’ 1696 0.004 0.01 22’ 14” 2371
Dragon 0 4096 27’ 53” 0.927 91% 30’ 9856 0.153 0.1 34’ 58” 8122
1 4096 33’ 33” 0.889 90% 30’ 8256 0.153 0.1 41’ 44” 8139
Island
0 1024 34’ 30” 0.992 87% 30’ 960 0.007 0.01 39’ 36” 1277
1 1024 42’ 35” 0.992 81% 30’ 768 0.007 0.01 51’ 11” 1289
3 1024 46’ 05” 0.992 80% 30’ 704 0.008 0.01 54’ 16” 1310
5 1024 47’ 20” 0.992 79% 30’ 704 0.008 0.01 55’ 56” 1317
Flowers 0 256 03’ 19” 0.901 15% 10’ 832 0.271 0.5 4’ 01” 291
1 256 03’ 28” 0.899 12% 10’ 784 0.271 0.5 4’ 15” 291
Performance Table
35
Performance Table
36
Teaser Dragon
Island Flowers
Implementation Guidelines
Bidirectional Path Tracing 1. Propagation
Altered as path tracing for camera and light sub-paths 1. Integration
The connection step can ignore occlusion Only form full paths with coherent propagation history: for instance, a sub-path that scattered on C cannot be connected to one that skipped C
37
Implementation Guidelines
Photon Mapping 1. Propagation
Altered as path tracing for photons and gathering rays 1. Integration
Gathering must remain coherent with propagation history: for instance, photons that scattered on C cannot be gathered by rays that skipped C
38
Implementation Guidelines
Metropolis Light Transport The mutation set does not change Propagation and integration are addressed during mutation acceptance 1. Propagation
A mutated path is allowed to cross the surface of at most one caster C … 1. Integration
… If the resulting history is coherent: it should not scatter on C elsewhere
39