Real-time skin rendering on graphics...
Transcript of Real-time skin rendering on graphics...
![Page 1: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/1.jpg)
![Page 2: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/2.jpg)
Real-time skin rendering on graphics hardware
RealReal--time skin rendering time skin rendering on graphics hardwareon graphics hardware
Pedro V. SanderDavid Gosselin
Jason L. MitchellATI Research
![Page 3: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/3.jpg)
Skin shadingSkin shadingSkin shading
• Most lighting comes from sub-surface scattering
• Traditional Lambertian lighting model is designed for hard surfaces with no sub-surface scattering so it doesn’t work well for skin
![Page 4: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/4.jpg)
Rough cross sectionRough cross sectionRough cross section
Air
Epidermis
Dermis
Bone, muscle, guts, etc.
![Page 5: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/5.jpg)
Our objectiveOur objectiveOur objective
• Develop a simple, efficient skin rendering algorithm for a real-time demo using ATI hardware
• Must be very fast, as it is one of several rendering techniques used concurrently in the demo
• Results that approximate sub-surface scattering
![Page 6: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/6.jpg)
Texture-space lightingTextureTexture--space lightingspace lighting
• Render diffuse lighting into an off-screen texture using texture coordinates as position
• Blur the off-screen diffuse lighting• Read the texture back and add specular
lighting in a subsequent final pass• We used a bump map for the specular
lighting pass only
![Page 7: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/7.jpg)
Previous workPrevious workPrevious work
• From Realistic Human Face Rendering for “The Matrix Reloaded” @ SIGGRAPH 2003:
• Our results:
From Matrix: Reloaded sketch
Current skin in Real Time
![Page 8: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/8.jpg)
Previous workPrevious workPrevious work
• [Borshukov03]– Texture lighting-based method almost
everywhere– Used traditional ray tracing for areas where
light can pass all the way through (e.g. ears) • We cannot afford to do that in real-time
• [Mertens03]– Similar image-space real-time technique
using importance sampling of BSSRDF
![Page 9: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/9.jpg)
OutlineOutlineOutline
• Algorithm overview• Texture space lighting
– Blur operation– Dilation
• Shadows– Soft shadows– Translucent shadows
• Acceleration techniques– Early-Z culling
![Page 10: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/10.jpg)
Basic approachBasic approachBasic approach
Geometry
Light in Texture Space Blur
Back Buffer
Sample texture space light
![Page 11: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/11.jpg)
Standard lighting modelStandard lighting modelStandard lighting model
![Page 12: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/12.jpg)
Blurred lighting modelBlurred lighting modelBlurred lighting model
![Page 13: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/13.jpg)
Rendering to textureRendering to textureRendering to texture
• Light as a 3D model• Draw into texture
– Pass texture coordinates as “position” – Rasterizer does the unwrap
![Page 14: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/14.jpg)
Spatially varying blurSpatially varying blurSpatially varying blur
• Used to simulate the subsurface component of skin lighting
• Used a growable Poisson disc filter• Read the kernel size from a texture• Allows varying the subsurface effect
– Higher for places like ears/nose– Lower for places like cheeks
![Page 15: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/15.jpg)
Filter kernelFilter kernelFilter kernel
Center Sample
Outer Samples
• Stochastic sampling• Poisson distribution • Samples stored as 2D
offsets from center
Large BlurLarge Blur
Small Blur
![Page 16: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/16.jpg)
Blur kernel size map and blurred lit textureBlur kernel size map Blur kernel size map and blurred lit textureand blurred lit texture
Blur Kernel Blur Kernel Size MapSize Map
Texture Space Texture Space LightingLighting ResultResult
![Page 17: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/17.jpg)
DilationDilationDilation• Texture seams can be a problem
(unused texels, bilinear blending artifacts)• During the blur pass we need to dilate• Use alpha channel of off-screen texture• If any sample has 1.0 alpha, just copy the
sample with the lowest alpha
![Page 18: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/18.jpg)
Dilation ResultsDilation ResultsDilation Results
Without DilationWithout Dilation With DilationWith Dilation
![Page 19: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/19.jpg)
Rim lightRim lightRim light
• We wanted to further emphasize the light that bleeds through the skin when backlit
• Compute the dot product between the negative light vector and the view vector
• Multiply result by Fresnel term• Only shows up if there is a light roughly
“behind” the object
![Page 20: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/20.jpg)
+ =No Rim LightNo Rim Light
Just Rim LightJust Rim LightLighting + Rim LightLighting + Rim Light
Added rim light resultAdded rim light resultAdded rim light result
![Page 21: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/21.jpg)
ShadowsShadowsShadows
• Used shadow maps– Apply shadows during texture lighting– Get “free” blur
• Soft shadows• Simulates subsurface interaction• Lower precision/size requirements• Reduces artifacts
• Only doing shadows from one key light– If using multiple lights,
still same # of blur passes
![Page 22: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/22.jpg)
Shadow mapsShadow mapsShadow maps
• Create projection matrix to generate map from the light’s point of view– Used bounding sphere of head to ensure
texture space is used efficiently• Shadow map rendering pass:
Write depth into off-screen texture• On texture lighting pass:
Test depth values in pixel shader
![Page 23: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/23.jpg)
Basic approach + shadowsBasic approach + shadowsBasic approach + shadows
Geometry
Light in texture space Blur
Back Buffer
Sample texture space light
Generate shadow map
![Page 24: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/24.jpg)
Shadow map and shadowed lit textureShadow map and Shadow map and shadowed lit textureshadowed lit texture
Shadow Map (depth)Shadow Map (depth) Shadows in Texture SpaceShadows in Texture Space
![Page 25: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/25.jpg)
Results with shadowsResults with shadowsResults with shadows
![Page 26: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/26.jpg)
Shadows from translucent objectsShadows from Shadows from translucent objectstranslucent objects
• Algorithm:– Draw depth of opaque shadow geometry to
shadow buffer alpha channel– Aditively blend RGB of translucent shadow
geometry into shadow buffer RGB channels• Depth test on• Depth write off
– On texture lighting pixel shader: non-shadowed pixels are multiplied by the translucent RGB value
![Page 27: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/27.jpg)
Basic approach + shadowsBasic approach + shadowsBasic approach + shadows
Opaquegeometry
Light in texture space Blur
Back BufferSample texture space light
Generate shadow map
Transparentgeometry
Blend translucency into shadow map
![Page 28: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/28.jpg)
Shadow map for transparent shadowsShadow map Shadow map for transparent shadowsfor transparent shadows
Translucent shadow mapTranslucent shadow map
RGBRGB
Opaque shadow mapOpaque shadow map
![Page 29: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/29.jpg)
Translucent shadows resultsTranslucent shadows resultsTranslucent shadows results
Opaque ShadowsOpaque Shadows Translucent ShadowsTranslucent Shadows
![Page 30: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/30.jpg)
SpecularSpecularSpecular
• Use bump map for specular lighting• Per-pixel exponent• Need to shadow specular
– Can’t have shadowed region with specular lighting spots– Expensive to do yet another blur pass for shadows
• Use luminance of blurred diffuse texture – Modulate specular from shadowing light by luminance of
texture space light (which very low-frequency)– Darkens specular in shadowed areas but preserves
lighting in unshadowed areas• Not a limitation, just an optimization
– One could do a separate blur pass for shadows
![Page 31: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/31.jpg)
Specular shadow dim resultsSpecular shadow dim resultsSpecular shadow dim results
Specular without shadowsSpecular without shadows Specular with shadowsSpecular with shadows
![Page 32: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/32.jpg)
Using Early-Z for CullingUsing EarlyUsing Early--Z for CullingZ for Culling
• Testing z-buffer prior to pixel shader execution– Can cull expensive pixel shaders– Only applicable when pixel shader does not output depth
• This texture-space operation doesn’t need the z buffer for hidden surface removal
• Can store any value of Z buffer• Use Early-Z to cull computations
– Back face culling– Distance and frustum culling
• Set z buffer on lighting pass according to frustum, distance from viewer, and facing-ness of polygons
• Set the z test such that non-visible polygons fail Z test• Reduces cost of image-space blurs in regions that don’t
need it
![Page 33: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/33.jpg)
Back Face CullingBack Face CullingBack Face Culling
Over the shoulder view of RubyOver the shoulder view of Ruby
Back facing pixels culled using early-zBack facing pixels culled using early-z
![Page 34: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/34.jpg)
SummarySummarySummary• Simple and fast skin rendering algorithm• Texture-space blur with dilation• Soft shadows “for free”• Translucent shadows• Early-Z culling acceleration techniques
![Page 35: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/35.jpg)
AcknowledgementsAcknowledgementsAcknowledgements
• Thanks to Eli Turner for help with the artwork
• Thanks to Chris Brennan for the colored translucent shadows suggestion
![Page 36: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/36.jpg)
ReferencesReferencesReferences• [Borshukov03] Borshukov and Lewis.
Realistic Human Face Rendering for “The Matrix Reloaded”. Technical Sketches, SIGGRAPH 2003.
• [Mertens03] Mertens et al. Efficient Rendering of Local Subsurface Scattering.Pacific Graphics 2003.
![Page 37: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/37.jpg)
DemoDemoDemo
![Page 38: Real-time skin rendering on graphics hardwaredeveloper.amd.com/wordpress/media/2012/10/Sander_SkinSketchSli… · Realistic Human Face Rendering for “The Matrix Reloaded”. Technical](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0dcc3a7e708231d43c2313/html5/thumbnails/38.jpg)
Questions?Questions?Questions?
Questions?Questions?Questions?