Post on 18-Feb-2021
Projective Textures & Shadow Mapping
Prof. Aaron Lanterman School of Electrical and Computer Engineering
Georgia Institute of Technology
2
What is projective texturing? • An intuition for projective texturing
– The slide projector analogy
Source: Wolfgang Heidrich [99]
From Stanford CS448A: Real-Time Graphics Architectures lecture 11; see graphics.stanford.edu/courses/cs448a-01-fall
3
“Slide projector” - different locations
Images from C. Everitt, “Projective Texture Mapping,” http://www.nvidia.com/object/Projective_Texture_Mapping.html
4
Texture matrix
€
strq
"
#
$ $ $ $
%
&
' ' ' '
=
12
0 0 12
0 12
0 12
0 0 12
12
0 0 0 1
"
#
$ $ $ $ $ $ $
%
&
' ' ' ' ' ' '
LightFrustrum(projection)Matrix
"
#
$ $ $ $
%
&
' ' ' '
LightView(lookat)Matrix
"
#
$ $ $ $
%
&
' ' ' '
ModelingMatrix
"
# $
%
& '
x0y0z0w0
"
#
$ $ $ $
%
&
' ' ' '
From “The Cg Tutorial,” p. 252.
ProjectTexture – setup Shader "GPUXX/ProjectTexture" { ! Properties { ! _BaseTex ("Base RGB) Gloss (A)", 2D) = "white" {} ! _ProjTex ("Base RGB)", 2D) = "white" {} ! _SpotPower ("Spotlightiness", Range(0.01,1)) = 0.7! } !!!!uniform sampler2D _BaseTex; !uniform float4 _BaseTex_ST; !uniform sampler2D _ProjTex; !uniform float _SpotPower; !!uniform float4x4 _myProjectorMatrix; !uniform float3 _spotlightDir; !
. . .
ProjectTexture – structures struct a2v { ! float4 v: POSITION; ! float3 n: NORMAL; ! float2 tc: TEXCOORD0; !}; ! !struct v2f { ! float4 sv: SV_POSITION; ! float2 tc: TEXCOORD0; ! float3 vWorldPos: TEXCOORD1; ! float3 nWorld: TEXCOORD2; ! float4 vProjected: TEXCOORD3; !}; !
ProjectTexture – vertex program v2f vert_projecttexture(a2v input) { ! v2f output; ! output.sv = mul(UNITY_MATRIX_MVP, input.v); ! output.vWorldPos = mul(_Object2World, input.v).xyz; ! // To transform normals, we want to use the inverse ! // transpose of upper left 3x3 ! // Putting input.n in first argument is like ! // doing trans((float3x3)_World2Object) * input.n;! output.nWorld = ! normalize(mul(input.n, (float3x3) _World2Object)); ! output.vProjected = mul(_myProjectorMatrix, !
float4(output.vWorldPos,1));! output.tc = TRANSFORM_TEX(input.tc, _BaseTex); ! return output; !} !
ProjectTexture – fragment program float4 frag_projecttexture(v2f input) : COLOR { ! // Only use this shader with a point light ! float3 lightDir = normalize(_WorldSpaceLightPos0.xyz -
input.vWorldPos * _WorldSpaceLightPos0.w); ! float3 eyeDir = !
normalize(_WorldSpaceCameraPos.xyz - input.vWorldPos); ! // Renormalizing because the GPU's interpolator ! // doesn't know this is a unit vector ! float3 n = normalize(input.nWorld); ! float3 diff_almost = !
unity_LightColor0.rgb * max(0, dot(n, lightDir)); ! float spotlightEffect = ! pow(dot(normalize(_spotlightDir), -lightDir),_SpotPower * 128.0); ! diff_almost *= spotlightEffect; ! diff_almost *= tex2Dproj(_ProjTex,input.vProjected); !! float4 base = tex2D(_BaseTex, input.tc); ! float3 output = ! (diff_almost + UNITY_LIGHTMODEL_AMBIENT.rgb) * base.rgb; ! return(float4(output,1)); !} !
C# script to set up projector matrix using UnityEngine; !using System.Collections; !![ExecuteInEditMode] !!public class SetupTextureProjection : MonoBehviour { !
! private Matrix4x4 projectorMatrix; ! private Matrix4x4 offsetMatrix; ! private Camera myCamera; !! void Start () { ! Vector3 halfs = 0.5f * Vector3.one;! offsetMatrix = Matrix4x4.TRS(halfs, Quaternion.identity, halfs); ! myCamera = GetComponent (); ! } !
! void Update () { ! projectorMatrix = offsetMatrix * myCamera.projectionMatrix * !
myCamera.worldToCameraMatrix; ! // Quick & dirty; it would generally be better to set properties ! // in specific materials! Shader.SetGlobalMatrix("_MyProjectorMatrix", projectorMatrix); ! Shader.SetGlobalVector("_spotlightDir", transform.forward); ! } !} !
10
Watch out for reverse projection!
Images from C. Everitt, “Projective Texture Mapping,” developer.nvidia.com/object/Projective_Texture_Mapping.html
11
Dramatic shadow in 2K Games’ BioShock
From www.wired.com/gaming/gamingreviews/multimedia/2007/08/pl_bioshock?slide=16&slideView=6
12
The shadow mapping concept (1)
• Depth testing from the light’s point-of-view – Two pass algorithm
• First, render depth buffer from the light’s point-of-view – The result is a “depth map” or “shadow map” – Essentially a 2D function indicating the depth of
the closest pixels to the light – This depth map is used in the second pass
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
13
The shadow mapping concept (2)
• Shadow determination with the depth map – Second, render scene from the eye’s point-of-view – For each rasterized fragment
• Determine fragment’s XYZ position relative to the light
• This light position should be setup to match the frustum used to create the depth map
• Compare the depth value at light position XY in the depth map to fragment’s light position Z
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
14
The shadow mapping concept (3)
• The Shadow Map Comparison – Two values
• A = Z value from depth map at fragment’s light XY position • B = Z value of fragment’s XYZ light position
– If B is greater than A, then there must be something closer to the light than the fragment, implies fragment is shadowed
– If A and B are approximately equal, implies fragment is lit
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
15
Shadow mapping: 2D illustration (1)
light source
eye position
depth map Z = A
fragment's light Z = B
depth map image plane
eye view image plane, a.k.a. the frame buffer
The A < B shadowed fragment case
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
16
light source
eye position
depth map Z = A
fragment's light Z = B
depth map image plane
eye view image plane, a.k.a. the frame buffer
The A ≥ B unshadowed fragment case
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Shadow mapping: 2D illustration (2)
17
Note image precision mismatch!
The depth map could be at a different resolution from the framebuffer
This mismatch can lead to artifacts
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Shadow mapping: 2D illustration (3)
18
Visualizing shadow mapping (1) • A fairly complex scene with shadows
the point light source
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
19
• Compare with and without shadows
with shadows without shadows Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (2)
20
• The scene from the light’s point-of-view
FYI: from the eye's point-of-view again
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (3)
21
• The depth buffer from the light’s point-of-view
FYI: from the light's point-of-view again
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (4)
22
• Projecting the depth map onto the eye’s view
FYI: depth map for light’s point-of-view again
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (5)
23
• Projecting light’s planar distance onto eye’s view
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (6)
24
• Comparing light distance to light depth map
Green is where the light planar
distance and the light depth
map are approximately
equal
Non-green is where shadows should be
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (7)
25
• Scene with shadows
Notice how specular
highlights never appear
in shadows
Notice how curved surfaces cast shadows on each other
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
Visualizing shadow mapping (8)
26
Depth map bias issues
Too little bias, everything begins to shadow
Too much bias, shadow starts too far back
Just right
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
27
Shadow mapping: filtering example GL_NEAREST: blocky GL_LINEAR: antialiased edges
Low shadow map resolution used to heighten filtering artifacts
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392
28
Projective texturing with spotlight shadows • Use a spotlight-style projected texture to give shadow
maps a spotlight falloff
Slide from C. Everitt, “Shadow Mapping,” Powerpoint presentation, www.nvidia.com/attach/6392