UNREAL SUMMIT 2016

Lighting the Planetary World of Project A1

Hyunwoo KiLead Graphics Programmer

UNREAL SUMMIT 2016

A1

• New IP of Nexon

– High-End PC / AAA-quality visuals

– MOBA / Space Opera

– UE4 + @@@

– In Development

• Announced our development last month

UNREAL SUMMIT 2016

A1

• Talk about character rendering at last NDC 2016 talk

• This talk presents techniques for lighting the world of A1

– Used a test scene

– Not the game world

UNREAL SUMMIT 2016

World of A1

• Spherical planet

• Real-time day and night cycle

• Partially environment destruction

– Trees, buildings, etc.

• Partially terrain modification

– Craters, explosion, etc.

UNREAL SUMMIT 2016

Challenges

• Spherical coordinates

• Longitudinal time variation

• Time of day lighting changes

• Dynamic

– Moving Sun

– Destruction

– Modification

UNREAL SUMMIT 2016

Our Approach

• “Fully Dynamic” if possible

– Shadow maps, SSAO, SSR, SSIS Screen Space Inner Shadows, etc.

• Partially Precomputation and Relighting

– Global illumination, sky lighting and reflection environment

UNREAL SUMMIT 2016

Changes for Planet: 1

• Vector towards the sky

– Vary according to latitude and longitude

– Standard world: just (0, 0, 1)

– Planetary world: normalize(WorldPosition)

• Assuming (0, 0, 0) is the center of the world

• We call this ‘Planet Normal’

UNREAL SUMMIT 2016

Changes for Planet: 2

• Longitudinal time variation

– Like GMT

float ComputeGMTFromWorldPosition(float3 WorldPosition){

float Longitude = atan2(WorldPosition.y, WorldPosition.x) / PI * 0.5f + 0.5f; // [0, 1]float NormalizedGMT = frac(Frame.NormalizedDayTime – Longitude); // East to Westreturn abs(NormalizedGMT);

}

* Note: Frame.NormalizedDayTime = GMT+0 = [0. 1)

UNREAL SUMMIT 2016

Agenda

• Directional Light

• Global Illumination

• Sky Light

• Reflection Environment

UNREAL SUMMIT 2016

Directional Light

UNREAL SUMMIT 2016

Directional Light

• Symmetry of two directional lights

– Sun and Moon

• Movable mobility

– For dynamic scenes

– Real-time lighting

• Deferred rendering: opaque materials

• Forward+ rendering: transparent materials

UNREAL SUMMIT 2016

Two of Directional Lights

• Sun:

– Dominant light

• Moon:

– Night area

– Adding direct specular

UNREAL SUMMIT 2016

Two of Directional Lights

• Problem:

– Directional lights commonly affect all surfaces on the world

– Incorrect results

• Ex) Moonlight leaks in the daytime

• Slow (2X)

• Solution:

– Cull backside of the planet from the light

– Smoothly attenuate radiance at boundaries

UNREAL SUMMIT 2016

Time of Day Lighting

• Different time for each pixel

• Overriding light color by using a hand-painted texture in the shader

• Different methods for GI, sky light and reflection environment

– See further slides

UNREAL SUMMIT 2016

Sun Shadows

• Shadows have an important role to recognize time during game play

• Presented at my NDC 2016 talk

– Use UE4 implementation

• CSM + PCF

– Add improved PCSS

• To control shadow softness by time: only for 0 and 1 cascade splits

• Shadow normal offset: to remove Peter Panning

• Temporal reprojection: to reduce flickering due to slow moving

UNREAL SUMMIT 2016

Tighter Shadow Bounds

• For both quality and speed

• Setting tighter bounds

– Assuming very far objects on the view do not cast shadows

– Backside of the planet culling + planetary view frustum culling

– More than 1.5X faster rendering

UNREAL SUMMIT 2016

Planetary View Frustum Culling

• Limit the far plane as distance between the camera and the center of the planet

– Assume that we can’t see backside of the planet

• Closer the camera, shorter the far plane

– A proportional expression betweenthe center of the planet and view frustum planes

UNREAL SUMMIT 2016

Before

Backside culling

Backside culling+ View frustum culling

UNREAL SUMMIT 2016

Global Illumination

UNREAL SUMMIT 2016

Existing Solutions in UE4

• Lightmaps (X)

– Static, and high memory consumption

• LPV (X)

– Slow, and low quality

• DFGI (X)

– Slow, and not supporting skeletal meshes

• Indirect lighting cache

– Be possible!

UNREAL SUMMIT 2016

UE4 Indirect Lighting Cache

• SH irradiance volume

• Per-primitive caching

– 5x5 volume -> upload to the global volume texture atlas

– For movable components or preview / Update when the component is moved

• “Try to use volume ILC for all types of components in the scene”

– Including static components and terrain

UNREAL SUMMIT 2016

Lacks of Volume ILC

• Lighting discontinuity (a.k.a. seams)

• Low density at a large geometry

– Need size-dependent cache distribution

• High memory consumption and slow cache update

• High CPU costs

– Per-primitive computation on the render thread

– Need update cache if a primitive is moved or lighting is changed

• Unsuitable for our game

UNREAL SUMMIT 2016

Need of a New Method

• Keep using SH irradiance volume

• More efficient data structure

• Seamless

• Faster update (or no cache update)

• Time of day lighting changes

UNREAL SUMMIT 2016

Related Work

• Far Cry series

• Assassin Creed series

• Quantum Break

• TC: The Division

• …

UNREAL SUMMIT 2016

Deferred Cubic Irradiance Caching

• ‘Deferred’:

– As post processing: avoiding overdraw

– Faster development iteration: quick recompile shaders

– But use forward rendering for transparency

• ‘Cubic’:

– Exploiting cubemaps: fit to GPUs

– Cache placement on the world: seamless

– Faster addressing: using planet normal = normalize(WorldPosition)

UNREAL SUMMIT 2016

UNREAL SUMMIT 2016

UNREAL SUMMIT 2016

Time of Day Lighting

• All day light

– Affect all day

• Time of day light

– Limited by time span

– 12 time spans: 0, 2, 4, …, 20, and 22 hour

• Twelve SH irradiance volumes

– Interpolate lighting from nearest 2 volumes

– No cache update

UNREAL SUMMIT 2016

Overview

• Offline

– Cache placement

– Photon emission

– Irradiance estimation

• Run-time

– Cubemap caching

– SH lighting

UNREAL SUMMIT 2016

Offline: Cache Placement

• Based on texels of the cubemap

– N x N x 6

– 50 cm space

– Finding Z by using ray casting

UNREAL SUMMIT 2016

Offline: Cache Placement

• 2.5D cache placement

– Above the surfaces

– No multi-layers or indoor in our game

– Incorrect results for flying characters

• Multiple layered cubemaps?

UNREAL SUMMIT 2016

Offline: Photon Emission

• UE4 Lightmass: a photon mapping based light builder

• Store ‘time of day light index’ for deposited photons

• No changes for remainders

class FIrradiancePhotonData{

FVector4 PositionAndDirectContribution;FVector4 SurfaceNormalAndIrradiance;

int32 TimeOfDayLightIndex;};

UNREAL SUMMIT 2016

Offline: Irradiance Estimation

• Twelve SH irradiance volumes

– Sharing world position, bent normal and sky occlusion

– Difference irradiance by time spans

• Per-time span irradiance

– Indirect photon final gathering

• All day lights: always

• Time of day lights: filtering by its index

• Global sky occlusion and bent normal

UNREAL SUMMIT 2016

UNREAL SUMMIT 2016

Run-time: Cubemap Caching

• Once at loading time

• CPU irradiance cache -> GPU cubemaps

– half4 texture cube array

• 2nd Order SH: encoded on 3 textures (RGB)

• 12 time spans * 3 = 36 elements

• Misc.:

– Bent normal and sky occlusion: 1

– Average color and directional shadowing: 12 – for reflection environment relighting

• Average color is computed by integrating incident radiance from all directions

UNREAL SUMMIT 2016

Run-time: SH Lighting

• Every frame

• Texture addressing

– Coordinates: planet normal

– Array index: time of day light index + 12 * {0|1|2}

• SH2 diffuse lighting

– Interpolate radiance from nearest 2 time spans

• Total 6 times fetches of a texture cube array

UNREAL SUMMIT 2016

Run-time: SH Lighting

UNREAL SUMMIT 2016

Performance

• Light building

– Scene-dependent

– Costly 12 times final gathering but faster than lightmaps

• Rendering

– Scene-independent

• approximately 0.4 ms: GTX970 1080p / ignoring transparency

• Stable visuals

– No seams or flickering

– Consistent looks for characters and environment

UNREAL SUMMIT 2016

Sky Light

UNREAL SUMMIT 2016

Transform to Spherical World

• Sky vector

– Not (0, 0, 1)

– Planet normal = normalize(WorldPosition)

• Rotate planet normal to (0, 0, 1) basis

– Heavy ALUfloat3 TransformVectorToLandsphereSpace(float3 InVector, float3 WorldPosition){

float3 PlanetNormal = normalize(WorldPosition);float3 SkyUp = float3(0, 0, 1);float3 RotationAxis = normalize(cross(PlanetNormal, SkyUp));float RotationAngle = acos(dot(SkyUp, PlanetNormal));float3x3 Rotator = RotationMatrixAxisAngle(RotationAxis, RotationAngle);return mul(Rotator, InVector);

}

UNREAL SUMMIT 2016

Time of Day Lighting

• Multiple sky cubemaps generated by artists

• Interpolated lighting like GI

UNREAL SUMMIT 2016

Bent Normal and Sky Occlusion

• As large scale ambient occlusion

– RGB: bent normal

– A: sky occlusion

– One R8G8B8A8 cubemap (no time variation)

• Diffuse lighting

– Widen and soft: sqrt

• Specular lighting

– Narrow and sharp: Square

Sky Occlusion + Bent Normal

UNREAL SUMMIT 2016

Screen Space Inner Shadows

• For GI and sky lighting

• Better looks when a character is on shadowed surfaces

• SSR styled ray marching

– Tracing on scene depth

– Directionality rather than SSAO

– No precomputation or asset building

– See my NDC 2016 presentation

UNREAL SUMMIT 2016

Reflection Environment

UNREAL SUMMIT 2016

Capture Probe Placement

• Uniform distribution on the sphere as the base

– Using golden spiral

• Additional placement by artists

UNREAL SUMMIT 2016

Time of Day Lighting

• Use relighting instead of pre-capturing all day

– Capture the scene without lighting

– Relight probes with tweak

• Relighting

– Geometric properties

• Relighting position = world position + (reflection vector * capture radius * specular occlusion)

• Relighting normal = normalize(relighting position)

– Direct lighting: Sun and SH2 diffuse sky lighting

– Indirect lighting: deferred cubic irradiance caching

• RGB: irradiance = average color of GI

• A: directional shadowing = occlusion of light sources

UNREAL SUMMIT 2016

UNREAL SUMMIT 2016

Volumetric Lighting

• Average color of GI (irradiance) is

also used for volumetric lighting

• For each ray marching step (for high spec.)

or at the surface (for low spec.)

UNREAL SUMMIT 2016

Summary

• Spherical world of A1

– Partially dynamic

– Time of day lighting changes

• Different approaches

– Deferred cubic irradiance caching

– Twelve time spans

– Relighting

UNREAL SUMMIT 2016

Future Work

• Improved GI

– 2nd Order SH -> 3th Order SH

– Layered cubemaps

– Volumetric fog

UNREAL SUMMIT 2016

Future Work

• Environment destruction and modification

– Multiple versions of irradiance volumes

• Pre-build for destroyed scenes

• Run-time update of cubemaps

• Like Quantum Break did

– Real-time GI

• SSGI?

– Recapture reflection environment

UNREAL SUMMIT 2016

Thank you

WE ARE HIRING!