Post on 13-Apr-2017
The Lightspeed Automatic Interactive Lighting Preview System
Jonathan Ragan-Kelley / MIT CSAILCharlie Kilpatrick / ILM
Brian Smith / ILMDoug Epps / Tippett Studio
Paul Green / MIT CSAILChristophe Héry / ILM
Frédo Durand / MIT CSAIL
Lighting Design
■End of pipelinefixed geometry, viewpoint, material
■Slow feedback10-60 mins to render
1 hour later...
Lighting Design
■End of pipelinefixed geometry, viewpoint, material
■Slow feedback10-60 mins to render
another hour later...
Goal: Fast Lighting Preview
Exploit redundancy between previews■ geometry■ view■material
High-Level Approach
■Precompute deep-framebuffer cache
■Preview dynamically on GPUas the user specifies new light parameters
...e.g.
normal position diffusetexture
speculartexture
Prior Work
Parameterized Ray Tracing [Séquin & Smyrl 1989]
G-Buffer [Saito & Takahashi 1990]
Fast Relighting Engine [Gershbein & Hanrahan 2000]
■Precomputed buffer(normals, texture)with BRDF*light reevaluated at runtime
■Shortcomings■ Simplistic shading■No antialiasing, motion blur, transparency
Render Caching
Pixar’s Lpics
■Production shaders& lights (RenderMan)
■Exploit programmable GPUs
■No antialiasing, transparency
■Requires extra shader-writing work■ (final) RenderMan version
+ extra caching code
■GPU preview version
[Pellacini et al. 2005]
Specializing Shaders
■Given dynamic parameters, automatically split shader code into:■ static■ dynamic
■Simple languageno control flow
■Simple shaders
[Guenter et al. 1995]
8
Bibliography [1] Andersen, Lars Ole. Self-applicable C Program
Specialization. Proceedings of the ACM SIGPLAN
Workshop on Partial Evaluation and Semantics-Based
Program Manipulation (San Francisco, California, June 12-
20, 1992). Yale University technical report
YALEU/DCS/RR-909, 1992, 54-61.
[2] Andersen, Peter Holst. Partial Evaluation Applied to Ray
Tracing. Unpublished manuscript, October 1994.
[3] Baier, Romana, Robert Glück, and Robert Zöchling. Partial
Evaluation of Numerical Programs in Fortran. Proceedings
of the ACM SIGPLAN Workshop on Partial Evaluation and
Semantics-Based Program Manipulation (Orlando, Florida,
June 25, 1994). University of Melbourne technical report
94/9, 1994, 119-132
[4] Demers, Alan J., Thomas Reps, and Tim Teitelbaum.
Incremental Evaluation for Attribute Grammars with
Application to Syntax-directed Editors. Proceedings of the
Eighth Annual ACM Symposium on Principles of
Programming Languages (Williamsburg, Virginia, January
1981), 105-116.
[5] Goad, Chris. Special Purpose Automatic Programming for
Hidden Surface Elimination. Proceedings of SIGGRAPH 82
(Boston, Massachusetts, July 26-30, 1982). In Computer
Graphics 16, 3 (July 1982), 167-178.
[6] Hanrahan, Pat. Ray Tracing Algebraic Surfaces.
Proceedings of SIGGRAPH 83 (Detroit, Michigan, July 25-
29, 1983). In Computer Graphics 17, 3 (July 1983), 83-90.
[7] Hanrahan, Pat and Jim Lawson. A Language for Shading
and Lighting Calculations. Computer Graphics 24, 4
(August 1990), 289-298.
[8] Hoover, Roger. Alphonse: Incremental Computation as a
Programming Abstraction. Proceedings of the SIGPLAN
‘92 Conference on Programming Language Design and
Implementation (San Francisco, California, June 1992),
261-272.
[9] Jones, Neil D., Carsten K. Gomard, and Peter Sestoft.
Partial Evaluation and Automatic Program Generation.
Prentice-Hall, 1993.
[10] Mogensen, Torben. The Application of Partial Evaluation to
Ray-Tracing. Master's thesis, DIKU, University of
Copenhagen, Denmark, 1986.
[11] Osgood, Nathaniel David. PARTICLE: an Automatic
Program Specialization System for Imperative and Low-
level Languages. Master's thesis, MIT, September 1993.
[12] Pugh, William and Tim Teitelbaum. Incremental
Computation Via Function Caching. Proceedings of the
Sixteenth Annual ACM Symposium on Principles of
Programming Languages (Austin, Texas, January 1989),
315-328.
[13] Ruf, Erik. Topics in Online Partial Evaluation. Ph.D.
thesis, Stanford University, April 1993. Published as
Stanford Computer Systems Laboratory technical report
CSL-TR-93-563, March 1993.
[14] Sequin, Carlo H. and Eliot K. Smyrl. Parameterized Ray
Tracing. Proceedings of SIGGRAPH 89 (Boston,
Massachusetts, July 31-August 4, 1989). In Computer
Graphics 23, 3 (July 1989), 307-314.
[15] Sundaresh, R.S. and Paul Hudak. A Theory of Incremental
Computation and Its Application. Proceedings of the
Eighteenth Annual ACM Symposium on Principles of
Programming Languages (Orlando, Florida, January 1991),
1-13.
[16] Upstill, Steve. The RenderMan Companion. Addison-
Wesley, 1989.
[17] Watkins, Christopher D., Stephen B. Coy, and Mark Finlay.
Photorealism and Ray Tracing in C. M&T Books, 1992
[18] Watt, Alan and Mark Watt. Advanced Animation and
Rendering Techniques. ACM Press, 1992.
Plate 1: Test image.
Plate 2: Image obtained by reshading image of Plate 1. This
took 12 seconds unspecialized, 0.5 seconds specialized.
8
Bibliography [1] Andersen, Lars Ole. Self-applicable C Program
Specialization. Proceedings of the ACM SIGPLAN
Workshop on Partial Evaluation and Semantics-Based
Program Manipulation (San Francisco, California, June 12-
20, 1992). Yale University technical report
YALEU/DCS/RR-909, 1992, 54-61.
[2] Andersen, Peter Holst. Partial Evaluation Applied to Ray
Tracing. Unpublished manuscript, October 1994.
[3] Baier, Romana, Robert Glück, and Robert Zöchling. Partial
Evaluation of Numerical Programs in Fortran. Proceedings
of the ACM SIGPLAN Workshop on Partial Evaluation and
Semantics-Based Program Manipulation (Orlando, Florida,
June 25, 1994). University of Melbourne technical report
94/9, 1994, 119-132
[4] Demers, Alan J., Thomas Reps, and Tim Teitelbaum.
Incremental Evaluation for Attribute Grammars with
Application to Syntax-directed Editors. Proceedings of the
Eighth Annual ACM Symposium on Principles of
Programming Languages (Williamsburg, Virginia, January
1981), 105-116.
[5] Goad, Chris. Special Purpose Automatic Programming for
Hidden Surface Elimination. Proceedings of SIGGRAPH 82
(Boston, Massachusetts, July 26-30, 1982). In Computer
Graphics 16, 3 (July 1982), 167-178.
[6] Hanrahan, Pat. Ray Tracing Algebraic Surfaces.
Proceedings of SIGGRAPH 83 (Detroit, Michigan, July 25-
29, 1983). In Computer Graphics 17, 3 (July 1983), 83-90.
[7] Hanrahan, Pat and Jim Lawson. A Language for Shading
and Lighting Calculations. Computer Graphics 24, 4
(August 1990), 289-298.
[8] Hoover, Roger. Alphonse: Incremental Computation as a
Programming Abstraction. Proceedings of the SIGPLAN
‘92 Conference on Programming Language Design and
Implementation (San Francisco, California, June 1992),
261-272.
[9] Jones, Neil D., Carsten K. Gomard, and Peter Sestoft.
Partial Evaluation and Automatic Program Generation.
Prentice-Hall, 1993.
[10] Mogensen, Torben. The Application of Partial Evaluation to
Ray-Tracing. Master's thesis, DIKU, University of
Copenhagen, Denmark, 1986.
[11] Osgood, Nathaniel David. PARTICLE: an Automatic
Program Specialization System for Imperative and Low-
level Languages. Master's thesis, MIT, September 1993.
[12] Pugh, William and Tim Teitelbaum. Incremental
Computation Via Function Caching. Proceedings of the
Sixteenth Annual ACM Symposium on Principles of
Programming Languages (Austin, Texas, January 1989),
315-328.
[13] Ruf, Erik. Topics in Online Partial Evaluation. Ph.D.
thesis, Stanford University, April 1993. Published as
Stanford Computer Systems Laboratory technical report
CSL-TR-93-563, March 1993.
[14] Sequin, Carlo H. and Eliot K. Smyrl. Parameterized Ray
Tracing. Proceedings of SIGGRAPH 89 (Boston,
Massachusetts, July 31-August 4, 1989). In Computer
Graphics 23, 3 (July 1989), 307-314.
[15] Sundaresh, R.S. and Paul Hudak. A Theory of Incremental
Computation and Its Application. Proceedings of the
Eighteenth Annual ACM Symposium on Principles of
Programming Languages (Orlando, Florida, January 1991),
1-13.
[16] Upstill, Steve. The RenderMan Companion. Addison-
Wesley, 1989.
[17] Watkins, Christopher D., Stephen B. Coy, and Mark Finlay.
Photorealism and Ray Tracing in C. M&T Books, 1992
[18] Watt, Alan and Mark Watt. Advanced Animation and
Rendering Techniques. ACM Press, 1992.
Plate 1: Test image.
Plate 2: Image obtained by reshading image of Plate 1. This
took 12 seconds unspecialized, 0.5 seconds specialized.
Precomputed Radiance Transfer
e.g. Sloan et al. 2002, Ng et al. 2003
Inappropriate for our application■ Large precomputation cost■ Largely ignore light functions■ Limitations on shading models
Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments Peter-Pike Sloan Jan Kautz John Snyder Microsoft Research ppsloan@microsoft.com
Max-Planck-Institut für Informatik jnkautz@mpi-sb.mpg.de
Microsoft Research johnsny@microsoft.com
Abstract We present a new, real-time method for rendering diffuse and glossy objects in low-frequency lighting environments that cap-tures soft shadows, interreflections, and caustics. As a preprocess, a novel global transport simulator creates functions over the object’s surface representing transfer of arbitrary, low-frequency incident lighting into transferred radiance which includes global effects like shadows and interreflections from the object onto itself. At run-time, these transfer functions are applied to actual incident lighting. Dynamic, local lighting is handled by sampling it close to the object every frame; the object can also be rigidly rotated with respect to the lighting and vice versa. Lighting and transfer functions are represented using low-order spherical harmonics. This avoids aliasing and evaluates efficiently on graphics hardware by reducing the shading integral to a dot product of 9 to 25 element vectors for diffuse receivers. Glossy objects are handled using matrices rather than vectors. We further introduce functions for radiance transfer from a dynamic lighting environment through a preprocessed object to neighboring points in space. These allow soft shadows and caustics from rigidly moving objects to be cast onto arbitrary, dynamic receivers. We demonstrate real-time global lighting effects with this approach.
Keywords: Graphics Hardware, Illumination, Monte Carlo Techniques,
Rendering, Shadow Algorithms.
1. Introduction
Lighting from area sources, soft shadows, and interreflections are important effects in realistic image synthesis. Unfortunately, general methods for integrating over large-scale lighting environ-ments [8], including Monte Carlo ray tracing [7][21][25], rad-iosity [6], or multi-pass rendering that sums over multiple point light sources [17][27][36], are impractical for real-time rendering.
Real-time, realistic global illumination encounters three difficul-ties – it must model the complex, spatially-varying BRDFs of real materials (BRDF complexity), it requires integration over the hemisphere of lighting directions at each point (light integration), and it must account for bouncing/occlusion effects, like shadows, due to intervening matter along light paths from sources to receiv-ers (light transport complexity). Much research has focused on extending BRDF complexity (e.g., glossy and anisotropic reflec-tions), solving the light integration problem by representing incident lighting as a sum of directions or points. Light integra-tion thus tractably reduces to sampling an analytic or tabulated BRDF at a few points, but becomes intractable for large light sources. A second line of research samples radiance and pre-convolves it with kernels of various sizes [5][14][19][24][34]. This solves the light integration problem but ignores light trans-port complexities like shadows since the convolution assumes the incident radiance is unoccluded and unscattered. Finally, clever techniques exist to simulate more complex light transport, espe-cially shadows. Light integration becomes the problem; these techniques are impractical for very large light sources.
Our goal is to better account for light integration and light trans-port complexity in real-time. Our compromise is to focus on low-
frequency lighting environments, using a low-order spherical harmonic (SH) basis to represent such environments efficiently without aliasing. The main idea is to represent how an object scatters this light onto itself or its neighboring space.
To describe our technique, assume initially we have a convex, diffuse object lit by an infinitely distant environment map. The object’s shaded “response” to its environment can be viewed as a transfer function, mapping incoming to outgoing radiance, which in this case simply performs a cosine-weighted integral. A more complex integral captures how a concave object shadows itself, where the integrand is multiplied by an additional transport factor representing visibility along each direction.
Our approach is to precompute for a given object the expensive transport simulation required by complex transfer functions like shadowing. The resulting transfer functions are represented as a dense set of vectors or matrices over its surface. Meanwhile, incident radiance need not be precomputed. The graphics hard-ware can dynamically sample incident radiance at a number of points. Analytic models, such as skylight models [33] or simple geometries like circles, can also be used.
By representing both incident radiance and transfer functions in a linear basis (in our case, SH), we exploit the linearity of light transport to reduce the light integral to a simple dot product between their coefficient vectors (diffuse receivers) or a simple linear transform of the lighting coefficient vector through a small transfer matrix (glossy receivers). Low-frequency lighting envi-ronments require few coefficients (9-25), enabling graphics hardware to compute the result in a single pass (Figure 1, right). Unlike Monte-Carlo and multi-pass light integration methods, our run-time computation stays constant no matter how many or how big the light sources, and in fact relies on large-scale, smooth lighting to limit the number of SH coefficients necessary.
We represent complex transport effects like interreflections and caustics in the transfer function. Since these are simulated as a preprocess, only the transfer function’s basis coefficients are affected, not the run-time computation. Our approach handles both surface and volume-based geometry. With more SH coeffi-cients, we can even handle glossy (but not highly specular) receivers as well as diffuse, including interreflection. 25 coeffi-cients suffice for useful glossy effects. In addition to transfer from a rigid object to itself, called self-transfer, we generalize the technique to neighborhood-transfer from a rigid object to its neighboring space, allowing cast soft shadows, glossy reflections, and caustics on dynamic receivers, see Figure 7.
Figure 1: Precomputed, unshadowed irradiance from [34] (left) vs. our precomputed transfer (right). The right model can be rendered at 129Hz with self-shadowing and self-interreflection in any lighting environment.
Our Design Goals
1.High-performance preview■ Low-latency feedback■ Fast initial precomputation
2.Seamless integration■ Same input: RenderMan scene & shaders
■ Same output: high quality image
3.Ease of implementation & maintenance
System Overview
System Overview
RenderMan scene & shaders
System Overview
RenderMan scene & shaders
cache & preview shaders
automaticcaching
System Overview
RenderMan scene & shaders
preview imagecache & preview shaders
relighting engine(gpu)
automaticcaching
System Overview
RenderMan scene & shaders
preview imagecache & preview shaders
relighting engine(gpu)
lighting parameters
iterative preview
automaticcaching
one-time precomputation
System Overview
RenderMan scene & shaders
preview imagecache & preview shaders
relighting engine(gpu)
lighting parameters
Automaticfor real production scenes‣ specialization & translation‣ cache compression
Automaticfor real production scenes‣ specialization & translation‣ cache compression
automaticcaching
System Overview
RenderMan scene & shaders
preview imagecache & preview shaders
relighting engine(gpu)
lighting parameters
Automaticfor real production scenes‣ specialization & translation‣ cache compression
High fidelityantialiasing, motion blur, transparency‣ indirect framebuffer
automaticcaching
Automatic Caching
RenderMan scene & shaders
cache & preview shaders
automaticcaching
preview image
relighting engine(gpu)
lighting parameters
Program Analysis
Program Analysis
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
Program Analysis
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
what part of the code depends on the lighting parameter L?
dynamic
Program Analysis
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
everything else is static
staticdynamic
Program Analysis
values at the boundary are cached for reevaluation
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
staticcache
dynamic
Program Analysis
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
Program Analysis
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
albedo
c Texture
U V
shade
Light BRDF
L N
Deep Framebuffer Generation
mySurface(colorc){albedo=c*Texture(U,V);shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}
Deep Framebuffer Generation
mySurfaceCache(colorc){albedo=c*Texture(U,V);cache(N);cache(albedo);
}
caching shader
Deep Framebuffer Generation
albedoN
mySurfaceCache(colorc){albedo=c*Texture(U,V);cache(N);cache(albedo);
}
deep framebuffer cache
caching shader
final renderer
Deep Framebuffer Generation
albedoN
deep framebuffer cache
caching shader
Deep Framebuffer Generation
albedoN
mySurfaceDynamic(vectorL){N=lookup();albedo=lookup();shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}preview shader
deep framebuffer cache
caching shader
Deep Framebuffer Generation
albedoN
mySurfaceDynamic(vectorL){N=lookup();albedo=lookup();shade=Light(L)*BRDF(N,L);returnalbedo*shade;
}preview shader
deep framebuffer cache
caching shader
Deep Framebuffer Generation
albedoN
preview shader
deep framebuffer cache
rendered imageshade
caching shader
Automatic Caching
Automatic Caching
RenderMan shaders
scene preprocessorRenderManscene
Automatic Caching
RenderMan shaders
scene preprocessor
program analysis
RenderManscene
Automatic Caching
RenderMan shaders
scene preprocessor
program analysis
RenderManscene
caching shaders
Automatic Caching
RenderMan shaders
scene preprocessor
final renderer
program analysis
RenderManscene
deep framebuffer cache
PN
x1x2
caching shaders
Automatic Caching
RenderMan shaders
scene preprocessor
dynamicpreview shaders
final renderer
program analysis
RenderManscene
deep framebuffer cache
PN
x1x2
caching shaders
Automatic Caching
RenderMan shaders
scene preprocessor
dynamicpreview shaders
final renderer
program analysis
RenderManscene
deep framebuffer cache
conservative
many cached terms
large caches
(hundreds)
(gigabytes)
program analysis is
generates
PN
x1x2
caching shaders
challenge: cache size
PN
x1x2
Cache Compression
scene preprocessor
final renderer
program analysis
Remove redundant values after caching
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
caching shaders
PN
x1x2
Cache Compression
scene preprocessor
final renderer
program analysis
Remove redundant values after caching
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
cache compression
PN
caching shaders
PN
x1x2
Cache Compression
scene preprocessor
final renderer
program analysis
Remove redundant values after caching
→ 5x compressionRenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
cache compression
PN
caching shaders
System Overview
scene preprocessor
final renderer
program analysis
cache comp.
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
PN
caching shaders
scene preprocessor
final renderer
program analysis
RenderMan shaders
RenderManscene
cache compr.
caching shaders
System Overview
scene preprocessor
final renderer
program analysis
cache comp.
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
PN
caching shaders
System Overview
preview image
relighting engine(gpu)
scene preprocessor
final renderer
program analysis
cache comp.
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
PN
caching shaders
lighting parameters
System Overview
preview image
relighting engine(gpu)
scene preprocessor
final renderer
program analysis
cache comp.
RenderMan shaders
dynamicpreview shaders
RenderManscene
deep framebuffer cache
antialiasing?transparency?
PN
caching shaders
lighting parameters
Classical Deep Framebuffer
A
B
Classical Deep Framebuffer
A
B
Classical Deep Framebuffer
PN
x1
x2
A
B
cache
shade
deep framebuffer cache
rendered image
Cache one sample per-pixelShade cache values to render image
Classical Deep Framebuffer
PN
x1
x2
A
B
cache
shade
How can we do antialiasing?
deep framebuffer cache
rendered image
PN
x1
x2
Classical Deep Framebuffer
A
B
cache
shade
How can we do antialiasing?transparency?
supersample image
Classical Deep Framebuffer
A
B
cache
shade
How can we do antialiasing?transparency?
PN
x3
x4supersample cache
supersample shading
Classical Deep Framebuffer
A
B
cache
shade
How can we do antialiasing?transparency?
PN
x3
x4
...?
...?
Our Indirect Framebuffer
■Decouple shading from visibility■ only supersample visibility
‣Do what RenderMan does
■Compress samplesbased on static visibility
RenderMan / REYES
A
B
show *filtering* pixel color?
RenderMan / REYES
b1
A
B
b2
a1
show *filtering* pixel color?
dice into micropolygons
RenderMan / REYES
b1
A
B
b2
a1
show *filtering* pixel color?
shade micropolygons
RenderMan / REYES
b1
A
B
b2
a1
show *filtering* pixel color?
rasterize micropolygons
RenderMan / REYES
b1
A
B
b2
a1
show *filtering* pixel color?
RenderMan / REYES
b1
A
B
b2
a1
1 2
3 4
a1
1
a1
3
a1
4
b2
2
show *filtering* pixel color?
a1pixel = +b2+a1+a1
RenderMan / REYES
b1
A
B
b2
a1
1 2
3 4
a1
1
a1
3
a1
4
b2
2
show *filtering* pixel color?
a1pixel =4
+b2+a1+a1
RenderMan / REYES
b1
A
B
b2
a1
1 2
3 4
a1
1
a1
3
a1
4
b2
2
show *filtering* pixel color?
a1pixel =4
+b2+a1+a1 = 0.75*a1+0.25*b2
RenderMan / REYES
b1
A
B
b2
a1
1 2
3 4
a1
1
a1
3
a1
4
b2
2
show *filtering* pixel color?
pixel =?
b1
RenderMan / REYES
a1
A
B
b2
pixel =?
b1
RenderMan / REYES
a1
A
B a1
α=0.3b1
α=1.0
1
0.3*a1+0.7*b1
b2
pixel =?
1
b1
RenderMan / REYES
a1
A
B a1
α=0.3b1
α=1.0
1
0.3*a1+0.7*b1
b2
α=1.0
2
1.0*b2
b2
pixel =?
1 2
b1
RenderMan / REYES
a1
A
B a1
α=0.3b1
α=1.0
1
0.3*a1+0.7*b1
a1
α=0.3
3
0.3*a1
a1
α=0.3b2
α=1.0
4
0.3*a1+0.7*b1
b2
α=1.0
2
1.0*b2
b2
pixel =?
1 2
3 4
b1
RenderMan / REYES
a1
A
B a1
α=0.3b1
α=1.0
1
0.3*a1+0.7*b1
a1
α=0.3
3
0.3*a1
a1
α=0.3b2
α=1.0
4
0.3*a1+0.7*b1
b2
α=1.0
2
1.0*b2
b2
pixel = 0.225*a1+0.35*b1+0.25*b2
1 2
3 4
Indirect Framebuffer
Indirect Framebufferb1
b2a1
Indirect Framebuffer
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x4
b1b2a1
no longer image space(per-micropolygon)
deepbuffer
Indirect Framebuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x4
b1b2a1
shade
deepbuffer
shaded like a conventional deep-framebuffer
Indirect Framebuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x4
b1b2a1
shade
indirect framebufferdeepbuffer
Indirect Framebuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x40.225 0.35 0.25
b1b2a1
shade
indirect framebufferdeepbuffer
densely stored in vertex array, on GPU
Indirect Framebuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x40.225 0.35 0.25
finalpixel
b1b2a1
shade
blend
indirect framebufferdeepbuffer
4 subpixel samples, 2 transparent layersonly 3 unique micropolygons
Indirect Framebuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x40.225 0.35 0.25
finalpixel
b1b2a1
shade
blend
indirect framebufferdeepbuffer
all contributions linearized into a single weight
4 subpixel samples, 2 transparent layersonly 3 unique micropolygons
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x40.242 0.331 0.245
finalpixel
b1b2a1
shade
blend
indirect framebuffer
More samples,Same cost
deepbuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x40.242 0.331 0.245
finalpixel
b1b2a1
shade
blend
indirect framebuffer
More samples,Same cost
deepbuffer
deepbuffer
shadingsamples
Px2
PP
x3
Nx1
x2
x3
∅ x4
b2b1a1
x4
finalpixel
b1b2a1
shade
blend
More samples,Same cost
0.242 0.331 0.245
indirect framebuffer
Indirect Framebuffer Resultsantialiasingmotion blurtransparencyidentical to RenderMan
Indirect Framebuffer Resultsantialiasingmotion blurtransparencyidentical to RenderMan
RenderMan45M micropolygons
60M subpixel samples
brute-force>100M samples
Indirect framebuffer 10M deep fb+15M indirect fb
720x389, 64x supersampling
System Architecture
System Architecture
RenderMan shaders
scene preprocessorRenderManscene
System Architecture
RenderMan shaders
scene preprocessor
program analysis
RenderManscene
System Architecture
RenderMan shaders
scene preprocessor
preview shaders
program analysis
RenderManscene
caching shaders
System Architecture
RenderMan shaders
scene preprocessor
preview shaders
final renderer
program analysis
RenderManscene
caching shaders
System Architecture
RenderMan shaders
scene preprocessor
preview shaders
final renderer
program analysis
cache compr.
RenderManscene
PN
x1x2
caching shaders
deep fb indirect fb
System Architecture
RenderMan shaders
relighting engine (GPU)scene preprocessor
preview shaders
final renderer
program analysis
cache compr.
RenderManscene
PN
x1x2
caching shaders
deep fb indirect fb
System Architecture
RenderMan shaders
relighting engine (GPU)scene preprocessor
preview shaders
final renderer
program analysis
cache compr.
RenderManscene
PN
x1x2
caching shaders shade
deep fb indirect fb
rendered image
System Architecture
RenderMan shaders
relighting engine (GPU)scene preprocessor
preview shaders
final renderer
program analysis
cache compr.
RenderManscene
PN
x1x2
caching shaders shade
deep fb indirect fb
blend
rendered image
System Architecture
RenderMan shaders
relighting engine (GPU)scene preprocessor
preview shaders
final renderer
program analysis
lightingparameters
cache compr.
RenderManscene
PN
x1x2
caching shaders shade
deep fb indirect fb
blend
iterative preview one-time precomputation
Additional Features
■Shadows
■Subsurface Scattering
■Performanceenhancement■ Progressive Refinement
■ Light Caching
■ Tiling…
Results
Lightspeed720x389
initial feedback: 0.05 secfull refinement: 0.7 sec
offline render: 5 mins
Results
LightspeedRenderMan720x389
initial feedback: 0.05 secfull refinement: 0.7 sec
offline render: 5 mins
Results
Results
914x38913x13 antialiasing42 lights
offline render: 59 mins
initial feedback: 0.11 secfull refinement: 2.7 sec
caching: 16 mins
Results
Lightspeed
RenderMan914x38913x13 antialiasing42 lights
offline render: 59 mins
initial feedback: 0.11 secfull refinement: 2.7 sec
caching: 16 mins
Results
Lightspeed
RenderMan914x38913x13 antialiasing42 lights
offline render: 59 mins
initial feedback: 0.11 secfull refinement: 2.7 sec
caching: 16 mins
Results
Results
Results
Results
Limitations
■GPU programming model■Dynamic calls to external C code■Complex data structures■GPU limits (bandwidth, memory, registers)
■ Fully-accurate dynamic ray tracing
■Unbounded dynamic loops
■Additional features not yet implemented
■ Indirect diffuse
■Deep shadows
■Non-linear lights
Discussion
■Automatic and manual specialization (Lpics) both have advantages■Manual specialization allows hand optimization■Compiler requires up-front R&D, never perfect
(especially on the GPU)
■ Saves significant time in production■ Some material parameters are editable
■Cache compression is key to practical automatic specializationmuch simpler than fancy static analysis
■ Indirect Framebuffer is powerful, scalable
Summary
■ Interactive lighting previewmilliseconds instead of hours
■Automatic caching for our production scenes■ Program analysis■Cache compression
■ Indirect framebuffer■ Efficient antialiasing, motion blur, transparency■ Progressive refinement
■ In use on current productions
Acknowledgments
Inception: Pat Hanrahan, Ujval Kapasi
Compilers: Alex Aiken, John Kodumal
Tippett: Aaron Luk, Davey Wentworth
ILM: Alan Trombla, Ed Hanway, Dan Goldman, Steve Sullivan, Paul Churchill
Images: Michael Bay, Dan Piponi
Money: NSF, NVIDIA, MSR, Sloan & Ford fellowships
Summary
■ Interactive lighting previewmilliseconds instead of hours
■Automatic caching for our production scenes■ Program analysis■Cache compression
■ Indirect framebuffer■ Efficient antialiasing, motion blur, transparency■ Progressive refinement
■ In use on current productions
EXTRAS
Cache Compression
Remove redundant values after caching■ non-varying
same for many pixels
■ non-uniquesame within a pixel
→ 4-5x compression
varying
unique
deep framebuffer cache
Cache Compression
shader dynamic(static analysis)
varying unique(compressed)
generic surface 402 145 97
metallic paint 450 150 97
Visibility Compression
RenderManIndirect
Framebuffer
scene res. samples shade subpix shade indirect
robot 914x389 13x13 2.1M 32M 633k 1.6M
robot (blur)
720x306 13x13 1.5M 21M 467k 3.8M
pirate 640x376 4x4 2.5M 2.3M 327k 716k
hairs 720x389 8x8 43M 58M 11M 17M