Around the World in 80 Shaders

Stephen McAuleyBizarre Creations

stephen.mcauley@bizarrecreations.com

Overview

• Introduction:– The FX system

• Around the world tour:– Default– Skin– MPEG corruption– Refraction mapping– Shallow water– Aquarium

FX System: Problem

• After finishing PGR4 and The Club we needed a new solution to handling shaders.– Two different systems.– Extremely complicated to add new shaders.• Have to add new vertex declarations, new exporter

code and new rendering code as well.

– Not data driven.• Shaders treated as code not data.• Could never run on an SPU.

– No Maya previewing for artists.

FX System: Solution

• Our solution was the FX system:– Based on .fx files.– Entirely data driven.– Automatic Maya previews for artists.– All rendering code now runs on the SPUs.

FX System: Maya and XSI

• Compile Maya and XSI shaders alongside PC shaders.

• Annotations control how the parameters appear to the artists:

• Artists can “expose” parameters they want to be set within game.

texture2D g_tAlbedo < string SasUiLabel = “Colour”; >;

float g_fScale < string SasUiLabel = “Scale”; > = 0.0;

FX System: Exporting

• Scene converter:– Reads vertex input structure• With the help of annotations.

– Exports relevant vertex data in the correct format.

struct C_VertexInput{ float3 m_vPosition : POSITION; float3 m_vNormal : NORMAL; float4 m_vTexCoord : TEXCOORD0 annotation(xy(uv:0) | zw(uv:luxlightmap) | format(F16_4));};

FX System: Exporting

• Lua script within each shader helps build shader permutations:

• If a bumpmap is present, game will search for ShaderB.fx instead of Shader.fx.

• Make sure you have compiled that permutation!

texture2D g_tBumpmap < string SasUiLabel = “Bumpmap”; >;

#if defined(FXFILE_PARSER)if (g_tBumpmap) then append(“B”, 10) end#endif

FX System: In Game

• Game sets exposed and shared parameters.• Techniques are changed dynamically:• e.g. shadow mapping technique, prepass technique…

• Shader parameters can be overridden via our debugging tool:– Currently only on all instances of a shader.

Default

Athens

Default Shader

• This is our base material for everything in game.

• Applied when artists use the Lambert shader in Maya.– Easy to set up.– Artists just set an albedo texture.– Other textures are picked up in the exporter using

a naming convention.

Default Shader: Material

• The following material options are supported:– Albedo (_DIFF)– Normal (_NORM)– Specular (_IPR0)– Emissive (_NEON)

• These are all controlled by textures.

Default Shader: Lighting

• We support these lighting methods:– Light maps– Vertex lighting– Vertex PRT– SH lighting

• In Blood Stone, they contained the following information:– Colour, sun occlusion and ambient occlusion

Default Shader: Lighting

Our BRDF for sun lighting:

where is the surface albedo and is Schlick’s fresnel approximation:

Default Shader: Specular

We have three parameters controlling specular:

• is the specular intensity• is the specular power• is the normal specular reflectance

Default Shader: Specular

• These three specular properties come from our specular texture:– Red channel: Intensity ()– Green channel: Power ()

– Blue channel: Reflectance ()

Skin

Istanbul

Skin

• Standard diffuse lighting gives unrealistic results when applied to skin.

• It looks hard and dry compared to skin’s soft appearance.

• To achieve this look, we need to model subsurface scattering.

Skin

• The best approach is described by Eugene d’Eon and David Luebke in [1]:– Observed that red light scatters in skin more than

green and blue.– Simulate subsurface scattering by lighting in

texture space, then performing separate gaussian blurs for red, green and blue channels.

• This is expensive and intrusive. Are there any cheaper ways?

Skin: Approximations

• Standard diffuse lighting:D = N.L

• Wrapped diffuse lighting:D = N.L * w + (1 – w)• i.e. D = N.L * 0.5 + 0.5

• What if we wrap different colours by different amounts?

Skin: Our solution

• Coloured wrapped diffuse lighting:D = N.L * W + (1 – W)

• i.e. D = N.L * { 0.5, 0.8, 0.9 } + { 0.5, 0.2, 0.1 }

• Simulates subsurface scattering by giving skin a softer look.

• Simulates red light scattering further in skin.• Cheap and easy to implement.

Standard Diffuse

Wrapped Diffuse

Coloured Wrapped Diffuse

MPEG Corruption

Monaco

MPEG Corruption

• For the AR phone, we wanted an effect that simulated MPEG corruption.

• “Blocks” in the image don’t update and remain from the previous frame.

MPEG Corruption

• Algorithm:– For each pixel, point sample a noise texture.• Scaled so each texel is the size of a “block”.

– If sample is less than a threshold, take pixel from previous frame buffer.• Instead of the previous frame buffer, use a scaled and

offset version of the current frame.

– Else, take pixel from current frame buffer.

MPEG Corruption

half2 vRUV = vUV * g_vCorruptionBlockSize; vRUV += half2(g_fCorruptionAmount * 7.123h, 0.0h);

half fRand = h4tex2D(g_tNoise, vRUV).r;

if(fRand < g_fCorruptionAmount){ real2 vDUV = input.m_TexCoord * g_vCorruptionUVScale; vDUV += g_vCorruptionUVOffset;

vColour = h4tex2D(g_tBackbuffer, vDUV).rgb; vColour *= (1.0h + fRand - 0.5h * g_fCorruptionAmount);}else{ vColour = h4tex2D(g_tBackbuffer, vUV).rgb;}

Refraction Mapping

Siberia

Refraction Mapping

• We wanted to simulate surfaces that are coated in a layer of ice.

• Requirements:– These surfaces have two layers:• Ice layer• Base layer

– The base layer is refracted by the ice layer.

Refraction Mapping

• Our in-game materials use three textures:– albedo– normal– specular

• We split these textures across the two layers.

Ice layer:• normal• specular

Base layer:• albedo

uv

refracted uv

Refraction Mapping

• Calculate the refracted texture coordinate:

• Albedo texture uses vRefractUV.• Normal and specular maps use vUV.

half3 vRefract = refract(vTangentView, vTangentNormal, 1.0f / g_fRefractiveIndex);

float2 vUVOffset = g_fDisplacement * vRefract.xy / vRefract.z;

float2 vRefractUV = vUV + vUVOffset;

Default

Displacement

Refraction

Refraction Mapping: Extensions

• Vary the displacement across the surface:– Height map.– Vertex colours.

• Add a frost layer in between the top layer and the base layer.

Default

Refraction

Frost

Refraction Mapping: Problems

• Make sure your tangent space is correct!– Any errors will very quickly become apparent.

• Displacement breaks as you look at the surface edge on.

Shallow Water

Burma

Shallow water

Deep water

Shallow Water

• Why does shallow water appear more transparent than deep water?– Water molecules absorb and scatter light.– The further light travels, the higher probability it is

absorbed or scattered.– Absorption and scattering are dependent on

wavelength:• e.g. red light is absorbed more than green or blue,

giving water its characteristic colour.

Shallow Water

• Premoze and Ashikhmin suggested the following model for the absorption and scattering of light in water [2]:

Shallow Water

𝐿=𝐿𝑧𝑒−𝑐𝑅+𝐿0(1−𝑒

−𝑐𝑅𝑒−𝐾 𝑑𝐻 )R

H

LZ

L0

extinction inscattering

c and Kd are wavelength dependent absorption and scattering coefficients

Shallow Water

• Optimisation:– Remove the term.– Only changes the falloff of the inscattering, not

the intensity.

Shallow Water

• Optimisation:– Remove the term.– Only changes the falloff of the inscattering, not

the intensity.

Shallow Water

• Optimisation:– Remove the term.– Only changes the falloff of the inscattering, not

the intensity.• Visual artefact:– Very shallow water is slightly darker.

Shallow Water

• The final equation is a lerp between the water colour and the underwater colour, based on the extinction:

• Final shader code:

half3 vExtinction = exp(-g_vExtinctionCoeffs * fDistUnderwater);

half3 vDiffuse = lerp(vSceneColour, vWaterColour, vExtinction);

No extinction

Red extinction

Green/blue extinction

Aquarium

Bangkok

Aquarium

• Is an aquarium tank equivalent to shallow water?

R LZ

L0

With Extinction

Image courtesy Jon Rawlinsonjonrawlinson.com

Aquarium

• Simulate a light positioned directly above the tank.

• This is our new inscattering term.

R LZ

L0

M0

H

MZ

Aquarium

• Inscattering equation:

• Light colour, , should be the colour of the light as it hits the surface of the water.– i.e. light colour modulated by water colour

• Use a different attenuation coefficient to the extinction.– Allows for more interesting hue shifts.

Aquarium

• Shader code:

half3 vExtinction = exp(-g_vExtinctionCoeffs * fDistanceThroughWater);

half3 vInscattering = exp(-g_vInscatteringCoeffs * fDistanceUnderWater);

vInscattering *= g_vLightColour.rgb;

half3 vDiffuse = lerp(vSceneColour, vInscattering, vExtinction);

Aquarium

• For the final touch, add light shafts:– Project two scrolling textures in the world x-z

plane.• Sample each texture at a fixed distance behind the

glass front of the aquarium tank.

– Combine texture samples into light shaft term, .– Modify inscattering term:

With Light

With Light Shafts

References

• [1] Advanced Techniques for Real-Time Skin Rendering, Eugene d’Eon and David Luebke, GPU Gems 3, 2008

• [2] Rendering Natural Waters, Simon Premoze & Michael Ashikhmin, 2001

Credits

• For working on the technology described in this presentation:– Paul Malin– Jan van Valburg– David Hampson