1. The Rendering Technologyof 'Lords of the Fallen'Philip Hammer (Deck13 Interactive)@philiphammer0phammer@deck13.comGameConnection Europe 2014October 29-31, Porte de Versailles, Paris

2. Agenda Introduction Current-Generation Deferred Rendering Deferred Transparent Lighting Transparent Shadowcasting Volumetric Lighting Q&AThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 3. Intro: Who is Deck13? Deck13 Interactive is one of the leading game developers in Germany ~50 people in Frankfurt office + 6 people in Hamburg office In-house multiplatform technology FLEDGE(PS4, Xbox One, PC, PS3, Xbox 360, iOS) Previously: Venetica, Jack Keane, Blood Knights, Tiger & Chicken, etc.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 4. Intro: Who am I? Senior Engine/Graphics Programmer Working since >8 Years @ Deck13 Interactive Part of an amazing team of 9 full-time programmers Responsible for the overall rendering system, lighting, materials, etc.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 5. Intro: What is Lords of the Fallen? Deck13s largest title so far 3rd-person challenging Action-RPG Dark, demonic fantasy setting Platforms: PS4, Xbox One, PC (DX11) Release this week First PS4 release from a german developer Some reviews are already outThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 6. Current Generation Deferred RenderingThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 7. Current Generation Deferred Rendering Were not doing super-duper funky stuff.. .. just combining some well-known techniques in the right way Were not covering the basics of deferred rendering.. .. but our specific differences to the textbook Game-production started as a PS3/X360 title Switched early in development after new platforms got introduced All last-gen assets replaced We had to find next gen for ourselves and adapt the techThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 8. Current Generation Deferred Rendering Major advancements in FLEDGE Switched from DX9 to DX11 only Switched from Light-Prepass to more traditional Deferred Rendering Support for deferred transparent object lighting (incl. particles) Support for Volumetric Lighting Support of parallax-corrected cubemap reflections [Lagarde12] Massively multithreaded rendering for consoles Massive utilization of hardware-instanced geometry Global Illumination (using Geomerics Enlighten) Advanced physics simulations for Clothing, GPU-Particles, etc.(using NVIDIA PhysX) Support a lot more objects, lights, shadows, decals, etc. etc.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 9. Current Generation Deferred Rendering Switch from Light-Prepass to Deferred Rendering Second geometry-pass was too costly Way more bandwidth and memory available in this HW generation(+ more flexible ESRAM on XB1) Challenge: Move a lot of rendering effects to deferred stages(e.g. Fog, Reflections, etc.)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 10. G-Buffer Layout G-Buffer should be as small as possible Parameter encoding 2-channel normals 2-channel albedo + specular-color(chroma subsampling, YCoCg, inspired by [Mavridis12]) Parameter packing Pack single bits and use lower precision for some attributes Share channels for mutually exclusive parameters Material-ID/-Index for referencing per-material parameters 128 bpp (31,64 MB @ 1080p) even fits fully into XB1 ESRAMThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 11. G-Buffer LayoutRED GREEN BLUE ALPHAMRT0RGBA8_UNORMEncoded Normal Material-The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014ID / IndexSpecularExponentMRT1RGBA8_UNORMAlbedoLumaAlbedoChromaSpec.LumaSpec. Chroma /TranslucencyL-Buffer (Diffuse)R11G11B10_FIrradiance Lightmap -Depth/StencilD24S8Depth Stencil 12. G-Buffer LayoutRED GREEN BLUE ALPHAMRT0RGBA8_UNORMEncoded Normal Material-The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014ID / IndexSpecularExponentMRT1RGBA8_UNORMAlbedoLumaAlbedoChromaSpec.LumaSpec. Chroma /TranslucencyL-Buffer (Diffuse)R11G11B10_FIrradiance Lightmap -Depth/StencilD24S8Depth Stencil Colored Specular and Translucency are mutually exclusive! Determined by Material-ID 13. G-Buffer LayoutRED GREEN BLUE ALPHAMRT0RGBA8_UNORMEncoded Normal Material-The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014ID / IndexSpecularExponentMRT1RGBA8_UNORMAlbedoLumaAlbedoChromaSpec.LumaSpec. Chroma /TranslucencyL-Buffer (Diffuse)R11G11B10_FIrradiance Lightmap -Depth/StencilD24S8Depth Stencil 1 Bit Material-ID determines whether MRT1.a containstranslucency or specular chroma. 7 Bit Material-Index used to referenceMaterial Parameter Buffers (more on this later) 14. G-Buffer LayoutRED GREEN BLUE ALPHAMRT0RGBA8_UNORMEncoded Normal Material-The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014ID / IndexSpecularExponentMRT1RGBA8_UNORMAlbedoLumaAlbedoChromaSpec.LumaSpec. Chroma /TranslucencyL-Buffer (Diffuse)R11G11B10_FIrradiance Lightmap -Depth/StencilD24S8Depth Stencil Stencil used for masking 3 Bits for Decals, Reflections, Skin/SSSSS 2 Bits double-sided Light-Volume Stenciling 15. G-Buffer Layout We needed more attributes for special effects Tangents for smooth alpha-test Per-object rimlights, emissive/glow & other FX Motion-Vectors of moving/dynamic objects for motionblur Distortion-Vectors for Post-Distort Not enough space in G-Buffer, but only needed for ~10% of objects Render them in a separate pass and blend (glow) oruse immediately in subsequent pass (motion-/distortion-vectors)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 16. Material Parameter Buffer G-Buffer stores surface-parameters .. .. but some surface-parameters are not needed per-pixel Store certain surface parameters indirectly to avoidwasting precious G-Buffer space Introduce a Material Parameter Buffer Generate a LUT which contains material parameters once per frame Store the LUT index in the G-Buffer to gain access to the parameters laterThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 17. Material Parameter Buffer Sparse parameters would unnecessarily bloat G-Buffer We dont want to afford more G-Buffer space We have quite a lot artist-tweakable parameters Material-based Parameter examples Reflection-Strength Fresnel Subsurface-scattering parameters etc.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 18. Material Parameter Buffer (Example) Example: Implementation ofApproximating Translucency for a Fast, Cheap andConvincing Subsurface Scattering Look [Brisebois12]in a deferred scenario without compromising the parameters Problem:A lot of properties varying per material, but effect is applied during indeferred light-pass Solution:Store scalar translucency values per-pixel in G-Buffer and all otherproperties in Material Parameter BufferThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 19. Material Parameter Buffer (Example)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014Material-Index sssDistortion sssPower sssMaterialColor 0 4.5 2.4 0.43, 0.39, 0.84 1 8.0 3.5 0.22, 0.51, 0.52 2 5.6 4.1 0.41, 0.43, 0.11 ... ... ... ... ...Parameters per LightParameters per MaterialParameters per Pixel// backscattering, according to [Brisebois12]float3 sssLightVector = L + N * sssDistortion ;float sssDot = exp2 ( saturate (dot ( V, -sssLightVector ) ) *sssPower - sssPower ) * sssScale ;float3 sssColor = ( sssDot *sssMaterialColor * sssLightColor ) * translucency ; 20. Material Parameter Buffer (Example)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014Material-Index sssDistortion sssPower sssMaterialColor 0 4.5 2.4 0.43, 0.39, 0.84 1 8.0 3.5 0.22, 0.51, 0.52 2 5.6 4.1 0.41, 0.43, 0.11 ... ... ... ... ...Find implementation details inAppendix 1 or ask me in Q&A. 21. Deferred Transparent LightingThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 22. Deferred Transparent Lighting Most engines use Forward Rendering for transparents in DeferredRendering scenarios Typical drawbacks: light-count, shadowing, etc. We wanted a deferred solution And we ended up with two different kinds of it Deferred Per-Pixel expensive - used for selected objects Deferred Per-Vertex cheap - used mostly for particlesThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 23. Deferred Per-Pixel Transparent Lighting Gives you one additional layer of lighting Straight-forward implementation(with similarities to Light-Prepass Rendering) 1. Render a second G-Buffer Only with lit transparent objects Advise the artists to flag only a few transparent objects as lit 2. Render a second L-Buffer Render all light volumes also on the second layer 3. Blend the transparent objects onto final scene whilesampling the light of the second L-BufferThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 24. Deferred Per-Pixel Transparent Lighting Caveats You *can* easily destroy your performance ,, .. but it wont if used wisely Memory consumption is higher, but not a big deal on current-gen console .. .. and you can reuse rendertargets in different rendering stages Optimizations Mask rendered pixels in stencil for culling Rough Scene-AABB culling Build AABB of all lit transparent objects once per frame Check against light-volume AABBs Omit a lot of lights cheaplyThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 25. Deferred Particle Lighting We want lit particles! Particles fitting better to the scene Particles can be re-used under different lighting conditionsThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 26. Deferred Particle Lighting Common type of particles: atmospheric, smoke, dust & fog effects Usually transparent, dense ... and cause a lot of overdraw Costs can easily explode when lighting every fragment Non-directional, low-frequency information Perfect for vertex-lighting Deferred vertex-lighting No compromise in number of affecting lights and shadows Fits perfectly into deferred pipelineThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 27. Deferred Particle Lighting Again three steps to success Step 1: Render Vertex G-Buffer Write particle vertex positions sequentially into a texture Step 2: Render Vertex L-Buffer Accumulate light volumes Step 3: Particle Rendering Sample L-Buffer and apply lighting Find implementation details in Appendix 2 and ask in Q&A.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 28. Deferred Particle LightingThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 29. Deferred Particle Lighting Great for many types of particles Cheap and effective Apply all light types and features like shadows, gobos, IES, etc. Inherits problems of all per-vertex techniques Low spatial resolution Works best with small particles Very small and bright lights can lead to flickering ifisolated corners of particles getting suddenly lit Be wary of buffer overflow with many lit particlesThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 30. Transparent ShadowsThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 31. Transparent Shadows All lights can cast colored, dynamic, transparent shadows Simple and effective approachThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 32. Transparent Shadows All lights can cast colored, dynamic, transparent shadows Simple and effective approach Render solid shadows as usual into shadowmap Render/Blend transparent objects into shadowmap-sized RGBA8 buffer Use shadowmap as depth-buffer Sample transparent shadowbuffer with shadow UVs and multiply with lightThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 33. Transparent Shadows Works also with volumetric lights, transparent particles, etc. Individual features really shine when everything works togetherThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 34. Volumetric LightingThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 35. Volumetric Lighting One of the major effect for Next Gen-Look Raymarching-based Approach Inspired by Real-time Volumetric Lighting in participating media [Toth09] Raymarching in lights view space while evaluating the shadowmap Interleaved Sampling and Temporal Reprojection for PerformanceThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 36. Volumetric Lighting Multiple light-types and -features Point, Spot, Box Also evaluate shadowmap, projector-texture(s), transparent shadows, IES light-profiles,animated noise volume-texture, etc.The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 37. Volumetric Lighting Optimizations Interleaved Sampling Low-resolution rendering with depth-aware upsampling Temporal ReprojectionThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 38. Volumetric Lighting PerformanceThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014Pass PC / DX11 (GTX700 GPU)PS4 / GNMAccumulation 0.362 ms 0.161 msGather 0.223 ms 0.375 msUpscale 0.127 ms 0.321 ms= 0.712 = 0.857 ms 39. Volumetric Lighting Attention: Artists love it and therefore tend to overuse it See for the gritty details:Volumetric Lighting for Many Lights in Lords of the Fallen by BenjaminGlatzelThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 40. Thanks for listening!@philiphammer0phammer@deck13.comThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 41. Thanks for listening!@philiphammer0phammer@deck13.comThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014Questions? 42. Thanks for listening!.. and thanks to the Deck13 development team, making all this possible... and very special thanks to the Tech-Team which contributed a lot to this talk (alphabetical) Arturo Cepeda Holger Durach Michele Giacalone (@miccalone) Benjamin Glatzel (@begla) Robert Hallinger (@robih29) Philip Hammer (@philiphammer0) Thorsten Lange Dominik Lazarek (@Omme) David Reinig (@D13_Dreinig)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 43. References[Glatzel14] Volumetric Lighting for Many Lights in Lords of the Fallen, Digital Dragons 2014,http://de.slideshare.net/BenjaminGlatzel/volumetric-lighting-for-many-lights-in-lords-of-the-fallen[Mavridis12] "The Compact YCoCg Frame Buffer", Journal of Computer Graphics Techniques, Vol. 1, No. 1, 2012[Lagarde12] Local Image-based Lighting With Parallax-corrected Cubemap, Siggraph, 2012[Brisebois12] Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look, GDC 2011http://dice.se/wp-content/uploads/Colin_BarreBrisebois_Programming_ApproximatingTranslucency.pdf[Persson07] Deep Deferred Shading, Blog Post 2007http://www.humus.name/index.php?page=3D&ID=75[Drobot12] "Lighting of Killzone: Shadow Fall"http://de.slideshare.net/guerrillagames/lighting-of-killzone-shadow-fall[Toth09] Real-time Volumetric Lighting in Participating Media, Eurographics 2009http://sirkan.iit.bme.hu/~szirmay/lightshaft.pdfThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 44. Appendix 1 - Material Parameter Buffer Step 1/3: Buffer generation Generate a buffer / LUT (array of structs) each frame on the CPU Determine for each drawcall a set of material parameters Smart value quantization and interleaving LUT contains selected data of the currently rendered materials DX11/GNM: structured buffer DX9: multiple 1D texturesThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 45. Appendix 1 - Material Parameter Buffer Step 2/3: G-Buffer stage Write the LUT-index to the G-Buffer Pass the index as shader constant(index is associated with the material during (1) ) can index up to 128 different materials per frame 7 bit, because 1 bit reserved for Material-IDThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 46. Appendix 1 - Material Parameter Buffer Step 3/3: Lighting/Combine stage(s) Bind the Material Parameter Buffer(s) to the shader Sample/unpack the Material-Index from G-Buffer Grab and use the parameter values when needed likevalue = buffer[index].xThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 47. Appendix 1 - Material Parameter BufferPixel-Shader// fragment shader for indexing deferred material-parameter BufferBuffer parameterBuffer ;uint materialIndex = unpackMaterialIndex(gbuffer0Value . a);float4 matParamBuffer0 = parameterBuffer[materialIndex];float4 matParamBuffer1 = parameterBuffer[materialIndex + 1];float sssDistortion = matParamBuffer0 . x ;// backscattering according to [Brisebois12]float3 sssLightVector = L + N * sssDistortion ;float sssDot = exp2 ( saturate ( dot ( V, -sssLightVector ) ) * sssPower - sssPower ) * sssScale ;float3 sssColor = ( sssDot * sssMaterialColor * sssLightColor ) * backscattering_mask ;The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 48. Appendix 2 - Deferred Particle Lighting Step 1/3: Particle Vertex G-Buffer Render a non-screenspace particle G-Buffer Basically a sequential list Render particle vertex-buffer as point-list(D3D11_PRIMITIVE_TOPOLOGY_POINTLIST) Rasterize viewspace-positions of particles subsequently into a buffer Currently a 1024x512 RGBA16F texture for all visible lit particles G-Buffer currently contains only vertex-positions Could be extended with more surface attributes liketranslucency or normalsThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 49. Appendix 2 - Deferred Particle Lighting Step 1/3: Particle Vertex G-Buffer// vertex-shader// pos = SV_Positionfloat4 pos = 1.0 ;// vertexId = SV_VertexID, instanceId = SV_InstanceID// param_vertexcount = shader constant holding the currently rendered number of verticesfloat vertexIndex = vertexId + ( instanceId * vertexCountPerInstance) + param_vertexcount ;float rasterPosY = trunc ( vertexIndex / textureWidth ) ;float rasterPosX = trunc ( vertexIndex - rasterPosY * textureWidth ) ;pos . x = ( ( rasterPosX * 2.0 ) - textureWidth ) / textureWidth ;pos . y = ( ( rasterPosY * 2.0 ) - textureHeight ) / textureHeight ;pos . zw = 1.0 ;The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 50. Appendix 2 - Deferred Particle Lighting Step 2/3: Particle Vertex L-Buffer Apply lighting similar to normal deferred lighting Use fullscreen-quads instead of light-volumes, because the fragmentshave no spatial relationship (non-screenspace fragment list) Optimization: Stencil-Mask each fragment in Per-Vertex G-Buffer Different light-types, shadows, projected textures, etc. just works Very simple lighting model: just add up light colors Shadowing: PCF with large kernel width reduces flickeringin shadow/light transitionsThe Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 51. Appendix 2 - Deferred Particle Lighting Step 2/3: Particle Vertex L-Buffer// pixel shader// trivial implementation (different for each light type):// - sample g-buffer containing the viewspace positions of particle vertices// - compute light attenuation base on position and light parameters (...)float3 positionVS = SAMPLE ( perVertexGBufferSampler, screenUV . xy ) . rgb ;col0 . rgb = light_color . rgb * getLightAttenuation ( positionVS ) * getShadowing ( positionVS ) ;The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 52. Appendix 2 - Deferred Particle Lighting Step 3/3: Particle Rendering Render particles as you would normally render them Apply lighting Compute UVs according to the rasterized position from Step 1 Sample the Particle L-Buffer in Vertex-Shader Interpolate lighting to fragment shader and modulate with particle texture Take care of exact rendering order(must be the same as in Step 1/3)The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014 53. Appendix 2 - Deferred Particle Lighting Step 3/3: Particle Rendering// vertex-shader// vertexId = SV_VertexID, instanceId = SV_InstanceID// param_vertexcount = shader constant holding the currently rendered number of verticesfloat vertexIndex = vertexId + ( instanceId * vertexCountPerInstance ) + param_vertexcount ;float3 perVertexLightBufferUV = 0;perVertexLightBufferUV . y = trunc ( vertexIndex / textureWidth ) ;perVertexLightBufferUV . x = trunc ( vertexIndex - perVertexLightBufferUV . y * textureWidth ) ;perVertexLightBufferUV . y = ( textureHeight - 1.0 ) - perVertexLightBufferUV . y ;float4 lbufferValue = perVertexLightBufferTexture . Load ( (int3)perVertexLightBufferUV . xyz ) ;// pixel-shadercol0 . rgb = particleTextureColor . rgb * f_in . lbufferValue . rgb ;The Rendering Technology of 'Lords of the Fallen', GameConnection Europe 2014