Post on 01-Dec-2018
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
GLOBAL ILLUMINATION IN UNITY 5
Jesper Mortensen
Software Engineer, Unity CPH
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
GLOBAL ILLUMINATION IN UNITY 5
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
GLOBAL ILLUMINATION IN UNITY 5
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
VIDEO – Unity Shrine by Alex Lovett
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
AGENDA
• What is Enlighten?
• How does Enlighten work?
• Realtime Reflections
• What’s next?
• Questions
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2014 @ UNITY TECHNOLOGIES
WHAT IS ENLIGHTEN?
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
WHAT IS ENLIGHTEN?
• Middleware for computing Global Illumination (GI)
• Developed by Geomerics/ARM based in Cambridge
• Supports all modern platforms (including mobile)
• Used for several successful AAA titles
• Enlighten 2.22 is integrated into Unity 5.0
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
ENLIGHTEN GI OUTPUTS
• Following a precompute phase
• Realtime lightmaps
• Realtime light probes
• Realtime cubemaps (coming in 5.x)
• Following an additional baking phase
• Baked lightmaps
• Baked light probes
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REALTIME GI OUTPUTS
• Realtime lightmaps/probes contain only indirect light
• Indirect light is generally low frequency
• Hence they can be low res
• It is radiosity only, so no specular bounce
• Light probes are L1 spherical harmonics
• Cubemaps for reflections – low res (coming in 5.x)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
BAKED GI OUPUTS
• High resolution baked lightmaps
• Direct lighting
• AO
• Indirect lighting
• Optional directional information for specular
• Relatively cheap
• But all lighting is static
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
WHAT DOES REALTIME GI MEAN
• Freely change all light properties in the player• Add/remove lights• Change light properties
• Freely modify material reflectance properties• Customized shader pass
• Freely change emission properties of static geometry• Customized shader pass
• Freely change the environment lighting (skybox)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
WHAT CAN YOU DO WITH THIS?
• Dynamic time of day• Glowy things• Attached lights
• Flashlights• Car headlights• Flaming swords• Lights that can be shot out• Explosions
• Mood lighting
Ghost of a Tale – Time of day tests in U5
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
Blacksmith – with indirect lighting
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
Blacksmith – without indirect lighting
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
WHAT CAN YOU *NOT* DO NOW
• Dynamic geometry cannot bounce light
• Dynamic geometry cannot emit light
• The majority of your scene must be static
• The precompute/bake phases are Editor only
• No fully procedurally generated worlds
• However, we intend to fix these in 5.x!
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
SMOKE AND MIRRORS™ – THE TRICKS• Add glow by attaching lights to dynamic objects
• Realtime transparency is upcoming in 5.x
• Support destruction with realtime transparency
• Support doors etc. with realtime transparency
• Use invisible emissive static geometry for glow
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2014 @ UNITY TECHNOLOGIES
HOW DOES ENLIGHTEN WORK?
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Enlighten has one purpose
• Compute indirect lighting fast
• It does that extremely well
• Everything else is built on top of that
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Solves an approximation to the rendering equation
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Assume finite elements
• Assume diffuse transport
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Well the radiosity of an element depends on the
radiosity of all the other elements!
• Expensive ~𝑂(𝑛2)• This won’t quite work…
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Observation 1: Make n small
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Lightmaps are low res and tightly packed
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Observation 2: Decouple emitters and receivers
• So n receivers and m emitters, where m < n
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Scene is clustered
Receivers (lightmap texels) Emitters (clusters)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Observation 3: mn matrix is sparse
• Distant clusters can be merged for a given texel
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Build cluster hierarchy
• Relaxation metric:
• Area
• Distance
• Normal
• Visibility
Scene root
Leaf clusters
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• For a given texel
• Select a hierarchy cut
• With a fixed budget
• Minimize error
Scene root
Leaf clusters
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Budget is a constant k
• Complexity is 𝑂(𝑛𝑘) ≈ 𝑂(𝑛)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Phase 1: Prime clusters with direct lighting
• We have a set of points per cluster
• Evaluate direct lighting for those
• Average to get per leaf cluster direct lighting
• Push this up the cluster hierarchy
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Phase 2: Read previously bounced lighting
• The cluster points has UVs mapping into output
• Read previous lighting value from output
• Average to get per leaf cluster bounced light
• Push this up the cluster hierarchy
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Phase 3: Gather
• For each pixel gather new lighting from cluster cut
• Done!
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
HOW DOES ENLIGHTEN WORK?
• Platform optimized runtimes
• Async from render loop
• Scheduling (coming in 5.x)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2014 @ UNITY TECHNOLOGIES
REALTIME REFLECTIONS
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REFLECTION PROBES
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REFLECTION PROBES
• Provide approximated specular reflection
• Component on a GameObject
• A box defines:
• the influence region
• the box projection [BEHC’10]
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REFL. PROBE CONVOLUTION
• Needed for rough surfaces
[LAMRUG]
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REALTIME REFL. PROBE CONVOLUTION• Runs on GPU in a pixel shader
• Seamless mipmap generation
• needs to support pre-DX10 hardware
• Accounts for projection
• sphere to cube
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REALTIME REFL. PROBE SCHEDULING• Time-slicing
• individual faces + 8 frames for convolution
• all faces at once + 8 frames for convolution
• all faces + convolution in one frame
• impacts framerate the most
• consistent results
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
SCREEN SPACE REFLECTIONS
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
SCREEN SPACE REFLECTIONS
• Image effect that extends built-in deferred shading
pipeline
• User-land scripts and shaders
• Adds mirror & glossy SSRR to emissive buffer
• Falls back to reflection probes on ray miss
• (coming in Unity 5.x)
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
SCREEN SPACE RAYTRACING
• Depth-buffer as proxy for scene geometry
• March the ray using the depth buffer
• At an intersection
• find corresponding value in the color buffer
• use as radiance
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REFLECTION PROBE ONLY
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
REFLECTION PROBE AND SSRR
COPYRIGHT 2014 @ UNITY TECHNOLOGIESCOPYRIGHT 2014 @ UNITY TECHNOLOGIES
WHAT’S NEXT?
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
WHAT’S NEXT?
• Fix bugs• Improve documentation• Optimize the precompute/bake phase
• LAN Lighting Build• Cloud Lighting Build
• Level up interation speed by integrating PowerVRtechnology
• Improve mixed realtime/baked shadows
COPYRIGHT 2014 @ UNITY TECHNOLOGIES
Text Styles
key to switch
between levels
24 pt.
Level 2 = Bullet 24 pt.
Level 3 = Bullet 20 pt.
Level 5 = Bullet 16 pt.
To get bullet / plain text
list
QUESTIONS?
● Global Illumination:
o @pigselated (Jesper Mortensen)
o @kengelstoft (Kasper Engelstoft)
o @kubacupisz
o @JoachimAnte
● Reflection probes:
o @__ReJ__ (Renaldas Zioma)
o Tomas Dirvanauskas
o Benoit Sévigny
o @dominiclaflamme
● Screen space reflections:
o @morgan3d (Morgan McGuire)
o @mikemx7f (Michael Mara)
● Blacksmith:
o S’Holm demo team
● Unity Shrine:
o @HeliosDoubleSix
● Ghost of a Tale:
o @SeithCG
● Dioramas:
o @yezzer
● Unity Labs:
o @David_Llewelyn
o @willgoldstone
o @peteorstrike (Peet Lee)