GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples •...
Transcript of GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples •...
![Page 1: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/1.jpg)
GPUGPU--AssistedAssistedRendering TechniquesRendering Techniques
Matthias M Wloka
![Page 2: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/2.jpg)
Shader Model 3.0 Hardware Is Here!
• What can I do with it?
• How can it help me with my game?– How can I use it to offload the CPU?– How can it help my image quality?
![Page 3: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/3.jpg)
How Can It Help My Game?
• Offload work from the CPU to the GPU– “Render To Vertex Buffer” technique– Uses texture-fetch feature of VS 3.0
• Image quality– Per-pixel specular lighting– Normal-map mipmaps produce artifacts
(distant objects shimmer)
![Page 4: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/4.jpg)
Offloading the CPU
Simulate Vertexpositions Render
to frame-buffer
Typical Workflow
![Page 5: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/5.jpg)
Offloading the CPU
CPU
Simulate Vertexpositions Render
to frame-buffer
Typical Workflow
GPU
![Page 6: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/6.jpg)
Simulating On the GPU?
GPU
Simulate Vertexpositions Render
to frame-buffer
GPU
Read-back: BAD!
• Use them programmable shaders!
~100s~100sMB/sMB/s
CPU
![Page 7: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/7.jpg)
“Render To Vertex Buffer”• Removes read-back from GPU to CPU
Render to texture
Store vertices as texture data
Read texture into vertex shader
Simulate Texture Renderto frame-buffer
GPU
~10s~10sGB/sGB/s
![Page 8: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/8.jpg)
Examples
• Cloth– Collide cloth against scene– Run cloth physics:
damped springs
• Displacement Mapping– Displace vertices
![Page 9: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/9.jpg)
More Examples• Snow/Sand accumulation
– Simulate friction/sliding
• Wind (simulation) bending grass
• Particle Systems
• Water waves/wakes
![Page 10: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/10.jpg)
Demo by Jeremy Zelsnack
![Page 11: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/11.jpg)
• Tessellated, flat plane for water
Rendering Water
Simulate Texture Renderto frame-buffer
GPU
Solve wave-equations
Store vertex heights
Read height in vertex shader
![Page 12: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/12.jpg)
How Does It Work?
• Create a vertex-mesh for water surface– Say, 128 by 128 vertices– Encode vertex’s mesh-
position as uv-coordinates
(0,0) (1,0)(.25,0) (.5,0) (.75,0)
![Page 13: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/13.jpg)
Vertex Shader Work
• Read ‘height-map’– Floating-point texture – Read texture at vertex’s uv
• Add result to vertex’s y
• Transform/Project vertex
vertex.y += tex(u, v)Out.pos = WorldViewProj * vertex
![Page 14: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/14.jpg)
Height-Map Is Dynamic
• Update every frame– With GPU via render-to-
texture
• Simulate water movementwith Verlet integration
![Page 15: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/15.jpg)
Verlet Integration
• A = ∑ (neighbors) – 4 Hn-1Hn = (Hn-1 – Hn-2) + A– Operates on positions only– No need to store velocity or acceleration
• Compute normal from positions: N = Normalize(S x T)
![Page 16: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/16.jpg)
Add Disturbances to Height Map
• Blend displacements into the water– For example: the boat, rocks, shore
• Verlet-integration integrates it next frame
• Yes, floating-point render-target blending
![Page 17: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/17.jpg)
Add Usual Eye Candy
• Caustics– Bilinear filtering of the normals crucial:
low-res (128x128) texture
• Reflection/Refraction
• Fresnel
![Page 18: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/18.jpg)
Advantages
• Fast!– Simulation happens on 128x128 texture – Small by GPU standards– Frame-rate unaffected by simulation
• Reasonable geometric complexity– 128x128 is 16k vertices
![Page 19: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/19.jpg)
More Details on This Sample(and Others)
• Next-Gen Special Effects Showcase– Wednesday,
12-1pm
![Page 20: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/20.jpg)
Particles via Render-to-VB• Building a Million
Particle System– Lutz Latta– Wednesday, 12-1pm
![Page 21: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/21.jpg)
Image Quality
• Per-pixel specular lighting– Normal-map mipmaps produce artifacts
(shimmer on distance objects)– Uses floating-point texture filter and blend
![Page 22: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/22.jpg)
Normal-Map Mipmap Artifacts
![Page 23: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/23.jpg)
What Is the Problem?• Specular term (N • H)s is high-frequency
function:1.0
-0.5 0.50.0
(N • H)40
![Page 24: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/24.jpg)
Sampling Frequencies
• Magnification case: accesses top-level mipmap– Sufficient sampling
• Minification case: lower mip-levels– Without mipmaps: sparkle city– With mipmaps: better, but not much
![Page 25: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/25.jpg)
What Is a Normal-Map Mipmap?
• Averaging– Replace 4 normals with
1 completely different normal
• Not re-normalizing shortens that normal– Scales down dot-product
shortensnormal
![Page 26: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/26.jpg)
What We Really Want
• N • H is a hack – N represents all normals in texel
• Integrate over all normals in texel– Integral is N • H only if
all N in texel are the same– Not true for mipmaps
![Page 27: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/27.jpg)
• Approximate dot-product via Gaussian:(N • H)s = cossα ≈ e-½ s α2
• Gaussians with standard deviation– Sum of them is another Gaussian with
standard deviation
How To Integrate
![Page 28: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/28.jpg)
After A Lot Of Math…
• Dot-product with Gaussian is:Na = ∑i Nift = 1/(1 + s (1/|Na| - 1)) (Toksvig factor)
specular term = ft ( (Na • H) / |Na|)ft s
![Page 29: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/29.jpg)
Corner Cases
• Dot-product with Gaussian is:Na = Nft = 1/(1 + s (1/|Na| - 1)) (Toksvig factor)
specular term = ft ( (Na • H) / |Na|)ft s
![Page 30: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/30.jpg)
Corner Cases
• Dot-product with Gaussian is:Na = Nft = 1/(1 + s (1/|N| - 1)) = 1
specular term = 1 ( (N • H) / |N|)1 s
= (N • H)
![Page 31: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/31.jpg)
Another Corner Case
• Dot-product with Gaussian is:Na = ∑i Ni = 0ft = 1/(1 + s (1/0 - 1)) = 0
specular term = 0 ( (0 • H) / 0)0 s
![Page 32: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/32.jpg)
Effect
• Length of normal expresses distribution
• Constant normal across texel – Computes sharp high-lights
• Widely varying normals across texel– High-light faints and widens
![Page 33: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/33.jpg)
Messy Formula To Compute?
• Function ft ( (Na • H) / |Na|)ft s depends on– s (constant)– Na • H– Na • Na
• 2D texture look-up: tex(Na • H, Na • Na)– Different 2D textures for different s
![Page 34: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/34.jpg)
Implementation
• Generate mipmaps via averaging – Leave vectors un-normalized!
• Fetch Na– Fp16 to minimize precision errors– Anisotropic filtering for best results
![Page 35: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/35.jpg)
Implementation Continued
• Compute Na • H and Na • Na
• Fetch specular using those coordinates– Since it is a function look-up:– Bilinear only– No mipmaps– 128x128 works ok
![Page 36: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/36.jpg)
Result and Observations• Short normals in base-level:
reduced specularity
• Specialize normal mipmapgeneration
• Applies to interp. vertex normals
• Applies to reflection-map lookups (LOD them)
![Page 37: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/37.jpg)
Before/After ComparisonBefore After
![Page 38: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/38.jpg)
Questions, Comments, Feedback?
• http://developer.nvidia.comThe Source for GPU Programming
• Slides available online
![Page 39: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/39.jpg)
More Rendering Techniques:NVIDIA’s SDK 7.0
• 200+ rendering techniques
• CD available @ NVIDIA’s booth
• Free
![Page 40: GPU-Assisted Rendering Techniquesdeveloper.download.nvidia.com/assets/gamedev/docs/...Examples • Cloth – Collide cloth against scene – Run cloth physics: damped springs • Displacement](https://reader033.fdocuments.in/reader033/viewer/2022060516/5fc618e9c96795208950d2a9/html5/thumbnails/40.jpg)
Other Rendering Technique Talks• Cinematic Effects II: The Revenge
– Wed, 9-10am
• GPU Gems Showcase– Wed, 5:30-6:30pm
• Real-Time TranslucentAnimated Objects– Fri, 2:30-3:30pm