Computer graphics & visualization The programmable (D3D 10) Pipeline.

computer graphics & visualization

Image Synthesis

The programmable(D3D 10) Pipeline


Image Synthesis – WS 07/08Dr. Jens Krüger – Computer Graphics and Visualization Group

Pixel Stage

Vertex Stage

User / Driver

First: the fixed function pipeline

Transform & Lighting Rasterizer

TexturingBlending/Ops

Texture 3Texture 2Texture 1Texture 0

Vertex Stream

Fragment Stream



Fixed Function Pipeline

• Properties

• States can be changed• States out of a fixed number of alternatives• Same principal operations always performed on stream elements• Can only discard certain operations• Action on stream elements (vertices/polygons/fragments) can not be programmed



Fixed Function Pipeline• To In the following:

– Non-OpenGL effects using fixed function pipeline– Multipass

• Combine pixel results of multiple rendering passes

– Pre-computed texture maps• Pre-compute results of certain non-standard operations

and store in texture maps



Advanced shadingThe Phong lighting model

– – N: surface normal– L: light source direction– R: reflection vector– V: direction to the viewpoint– n: materials specular reflection exponent

• Defines the highlights falloff from the direction of reflection

– Specular reflections show the color of the light source• Not valid for all materials

nsdpattaa RVkLNkIfkII



Advanced shadingThe Phong/Blinn lighting model

–

– The Halfway vector H points into the direction of a surfaces´ normal that reflects light into V

– If light source and view point are considered to be at infinity, H is a constant

– The angle between R and V is twice the angle between N and H if all above vectors are complanar

• Highlights appear sharper in the Phong model

nsdpattaa HNkLNkIfkII



Advanced shading• In OpenGL, per-vertex Phong lighting is

employed to simulate specular reflections• But, during scan-conversion vertex color is

interpolated – OpenGL uses Gouraud shading instead of Phong

shading– Texture color affects highlight color– Sharp highlights might be missed if the surface is

tesselated coarsely



Advanced shading• To get separate highlight color highlights have

to be added to the diffusely lit textured surface– OpenGL computes Ct(Cd + Cs)– Instead

• Compute Cd and Cs separately • Modulate Cd with Ct • Add result to Cs

– Solution can be obtained by -blending



Advanced shading• Multipass technique to get separate highlight

color– Render the textured object without specular light

CtCd

– Render the untextured object with specular light only Cs

– Combine the results in the frame buffer by additive blend

• glBlendFunc(GL_ONE, GL_ONE)• glBlendEquation(GL_ADD)



Advanced shading

Textured with diffuse light

Untextured with specular

light

Combined images

In the original image the

highlight can hardly be seen



Advanced shading• Using OpenGL 1.2

glLightModel(GL_LIGHT_MODEL_COLOR_CONTROL, GL_SEPARATE_SPECULAR_COLOR);



A few words on OpenGL Extensions• The concept of extensions

– Naming conventions• ARB vs. cross vendor vs. vendor specific extensions

– Querying:• compile time vs. run time



OpenGL Extensions• Mechanism for providing access to non-

standard features of the hardware• Every vendor may define own extensions

without having to ask anybody else• Multiple vendors may cooperate to define more

common extensions to ensure compatibility across platforms



Naming Conventions for Extensions• ARB: Architecture Review Board

– Ext. that is likely to become part of core OpenGL in the future

• EXT: Multi-vendor extensions– Ext. that multiple vendors have agreed upon

• NV, ATI, SGI, SUN...: vendor-specific ext.– Ext. supported only by one vendor, sometimes

experimental and not available on all systems of that vendor (e.g. NVX, SGIS, SGIX)



Dealing with Extensions• Since all extensions are optional, applications

cannot rely on their presence• Have to check both at compile time and at run

time!• Compile time check:

#ifdef GL_ARB_multitexture// use multitexture ext. in here...#endif



Dealing with Extensions (2)Run time check:

const Glubyte *extstring = glGetString(GL_EXTENSIONS);strcpy(exts, extstring);extsupported= 0;next= strtok(exts, “ “);while(next) {

if(!strcmp(next,“GL_ARB_multitexture”))extsupported= 1; break;

next= strtok(exts,NULL);}



Advanced shadingHow to get sharp highlights without adaptive refinement (tesselation)

– Using the Blinn/Phong model specular highlights depend on L,N and V

– By fixing L and V highlights depend only on N– N can be used to index into a texture map in which

the reflection for R = f(N,L) are stored– Texture map looks like reflection from chrome

hemisphere



Texture coordinates computed from the

normal vector

Texture coordinates map into sphere map

texture

Texel contain pre-computed Phong

lighting

Advanced shadingSpherical texture maps



Advanced shadingPhong shading with highlight texture

– Render the object with diffuse light CtCd

– Render the unlit, untextured white object and modulate it with the highlight texture Cht

– Combine the results in the frame buffer by additive blend

– … or use multi-textures in one rendering pass



Advanced shadingPhong shading with highlight texture

Textured with diffuse light

White object modulated by

highlight texture

Blended images



No more “Hacks” …The Programmable pipeline• Direct3D 9 / OpenGL 2.0 Model

– Still used in many existing games– Supported by any graphics card of the last 5 years– Supported by every mayor OS

(DX only on Windows)• Direct3D 10 / OpenGL 2.0 + Ext / OpenGL 3.0

– Supported only by the latest GPUsNVIDIA 8xxx or ATI Radeon 2xxxx

– OpenGL 3.0 not out yet must be used via extensions– Direct3D 10.0 only supported by Windows Vista

(Direct3D 10.1 requires Vista SP1)



The Direct3D 9.0c PipelineInput Data

Input AssemblerStage (IA)

Vertex ShaderStage (VS)

Rasterizer Stage(RS)

Pixel ShaderStage (PS)

Output MergerStage (OM)

Output Data

MemoryResources:

Buffers, TexturesBuffer

Texture, Constants

Texture , Constants

States

Texture



Features of the D3D 9.0c pipeline• Programmable vertex and pixel shader

with limited shader program length (and many other limitations)

• Vertex and pixel shader texture fetch capabilities (some limitations apply)

• Configurable output merger (Blending/Ops/Tests)• Fixed function pipeline vertex and/or pixel shader

can be used• Floating point through the entire pipeline (except

for the blending)



Benefits of the programmable Pipeline• No more “hacks” more sophisticated “hacks”• Straight forward program flow without the

necessity of cryptic configuration changes• Decent knowledge of the transformation and

lighting required to program the pipeline everything has to be done “by hand”

• Shaders can be programmed in a high level language HLSL/GLSL



Shaders: a first look (HLSL)“Basic HLSL”-Sample from the DirectX SDK://--------------------------------------------------------------------------------------// Global variables//--------------------------------------------------------------------------------------float4 g_MaterialAmbientColor; // Material's ambient colorfloat4 g_MaterialDiffuseColor; // Material's diffuse colorint g_nNumLights;

float3 g_LightDir[3]; // Light's direction in world spacefloat4 g_LightDiffuse[3]; // Light's diffuse colorfloat4 g_LightAmbient; // Light's ambient color

texture g_MeshTexture; // Color texture for mesh

float g_fTime; // App's time in secondsfloat4x4 g_mWorld; // World matrix for objectfloat4x4 g_mWorldViewProjection; // World * View * Projection matrix

//--------------------------------------------------------------------------------------// Texture samplers//--------------------------------------------------------------------------------------sampler MeshTextureSampler = sampler_state{ Texture = <g_MeshTexture>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};

//--------------------------------------------------------------------------------------// Vertex shader output structure//--------------------------------------------------------------------------------------struct VS_OUTPUT{ float4 Position : POSITION; // vertex position float4 Diffuse : COLOR0; // vertex diffuse color (note that COLOR0 is clamped from 0..1) float2 TextureUV : TEXCOORD0; // vertex texture coords };

//--------------------------------------------------------------------------------------// This shader computes standard transform and lighting//--------------------------------------------------------------------------------------VS_OUTPUT RenderSceneVS( float4 vPos : POSITION, float3 vNormal : NORMAL, float2 vTexCoord0 : TEXCOORD0, uniform int nNumLights, uniform bool bTexture, uniform bool bAnimate ){ VS_OUTPUT Output; float3 vNormalWorldSpace; float4 vAnimatedPos = vPos; // Animation the vertex based on time and the vertex's object space

position if( bAnimate )

vAnimatedPos += float4(vNormal, 0) * (sin(g_fTime+5.5)+0.5)*5;

// Transform the position from object space to homogeneous

projection space

Output.Position = mul(vAnimatedPos, g_mWorldViewProjection); // Transform the normal from object space to world space vNormalWorldSpace = normalize(mul(vNormal,

(float3x3)g_mWorld)); // normal (world space) // Compute simple directional lighting equation float3 vTotalLightDiffuse = float3(0,0,0); for(int i=0; i<nNumLights; i++ ) vTotalLightDiffuse += g_LightDiffuse[i] *

max(0,dot(vNormalWorldSpace, g_LightDir[i])); Output.Diffuse.rgb = g_MaterialDiffuseColor * vTotalLightDiffuse + g_MaterialAmbientColor * g_LightAmbient; Output.Diffuse.a = 1.0f; // Just copy the texture coordinate through if( bTexture ) Output.TextureUV = vTexCoord0; else Output.TextureUV = 0; return Output; }

struct PS_OUTPUT{ float4 RGBColor : COLOR0; // Pixel color };

PS_OUTPUT RenderScenePS( VS_OUTPUT In, uniform bool bTexture ) { PS_OUTPUT Output;

// Lookup mesh texture and modulate it with diffuse if( bTexture ) Output.RGBColor = tex2D(MeshTextureSampler, In.TextureUV) * In.Diffuse; else Output.RGBColor = In.Diffuse;

return Output;}

technique RenderSceneWithTexture1Light{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 1, true, true ); PixelShader = compile ps_2_0 RenderScenePS( true ); // trivial pixel shader (could use FF instead if desired) }}technique RenderSceneWithTexture2Light{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 2, true, true ); PixelShader = compile ps_2_0 RenderScenePS( true ); // trivial pixel shader (could use FF instead if desired) }}technique RenderSceneWithTexture3Light{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 3, true, true ); PixelShader = compile ps_2_0 RenderScenePS( true ); // trivial pixel shader (could use FF instead if desired) }}technique RenderSceneNoTexture{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 1, false, false ); PixelShader = compile ps_2_0 RenderScenePS( false ); // trivial pixel shader (could use FF instead if desired) }}



Vertex Shaders: a closer lookVS_OUTPUT RenderSceneVS( float4 vPos : POSITION, float3 vNormal : NORMAL, float2 vTexCoord0 : TEXCOORD0, uniform int nNumLights, uniform bool bTexture, uniform bool bAnimate ){ VS_OUTPUT Output; float3 vNormalWorldSpace; float4 vAnimatedPos = vPos; // Animation the vertex based on time and the vertex's object space position if( bAnimate ) vAnimatedPos += float4(vNormal, 0) * (sin(g_fTime+5.5)+0.5)*5; // Transform the position from object space to homogeneous projection space Output.Position = mul(vAnimatedPos, g_mWorldViewProjection); // Transform the normal from object space to world space vNormalWorldSpace = normalize(mul(vNormal, (float3x3)g_mWorld)); // normal (world space) // Compute simple directional lighting equation float3 vTotalLightDiffuse = float3(0,0,0); for(int i=0; i<nNumLights; i++ ) vTotalLightDiffuse += g_LightDiffuse[i] * max(0,dot(vNormalWorldSpace, g_LightDir[i])); Output.Diffuse.rgb = g_MaterialDiffuseColor * vTotalLightDiffuse + g_MaterialAmbientColor * g_LightAmbient; Output.Diffuse.a = 1.0f; // Just copy the texture coordinate through if( bTexture ) Output.TextureUV = vTexCoord0; else Output.TextureUV = 0; return Output; }



Pixel Shaders: a closer lookPS_OUTPUT RenderScenePS( VS_OUTPUT In, uniform bool bTexture ) { PS_OUTPUT Output;

// Lookup mesh texture and modulate it with diffuse if( bTexture ) Output.RGBColor = tex2D(MeshTextureSampler, In.TextureUV) *

In.Diffuse; else Output.RGBColor = In.Diffuse;

return Output;}



Putting it all together: Effect Files• Store many vertex and pixel shaders as well as

pipeline stages in a single file• Also Store variables, textures, and samplers• Allow for grouping of shaders into techniques and

passes• Can be accessed easily from the main program• Can be created/modified easily from a content

creation tool(AMD RenderMonkey/Nvidia FX Composer)

http://ati.amd.com/developer/rendermonkey/index.html

http://ati.amd.com/developer/rendermonkey/index.html

http://developer.nvidia.com/object/fx_composer_home.html



FX Composer Showcase



Effects in the “Basic HLSL”-Sample //--------------------------------------------------------------------------------------// Global variables//--------------------------------------------------------------------------------------float4 g_MaterialAmbientColor; // Material's ambient colorfloat4 g_MaterialDiffuseColor; // Material's diffuse colorint g_nNumLights;

float3 g_LightDir[3]; // Light's direction in world spacefloat4 g_LightDiffuse[3]; // Light's diffuse colorfloat4 g_LightAmbient; // Light's ambient color

float g_fTime; // App's time in secondsfloat4x4 g_mWorld; // World matrix for objectfloat4x4 g_mWorldViewProjection; // World * View * Projection matrix



Effects in the “Basic HLSL”-Sample texture g_MeshTexture; // Color texture for mesh

//--------------------------------------------------------------------------------------// Texture samplers//--------------------------------------------------------------------------------------sampler MeshTextureSampler = sampler_state{ Texture = <g_MeshTexture>; MipFilter = LINEAR; MinFilter = LINEAR; MagFilter = LINEAR;};



Effects in the “Basic HLSL”-Sample //--------------------------------------------------------------------------------------// Renders scene to render target//--------------------------------------------------------------------------------------technique RenderSceneWithTexture1Light{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 1, true, true ); PixelShader = compile ps_2_0 RenderScenePS( true );

// trivial pixel shader (could use FF instead if desired) }}



Effects in the “Basic HLSL”-Sample technique RenderSceneWithTexture2Light{ pass P0 { VertexShader = compile vs_2_0 RenderSceneVS( 2, true, true ); PixelShader = compile ps_2_0 RenderScenePS( true );

// trivial pixel shader (could use FF instead if desired) }}



The DX 10.0 PipelineInput Data

Input AssemblerStage (IA)

Vertex ShaderStage (VS)

Geometry ShaderStage (GS)

Stream OutputStage (SO)

Rasterizer Stage(RS)

Pixel ShaderStage (PS)

Output MergerStage (OM)

Output Data

Memory Resources:

Buffers, Textures,

Constant Buffers

Buffer

Texture, Constant Buffer



Buffer

States

Buffer, Texture, Constant Buffer



Features of the D3D 10.1 pipeline• Programmable vertex, pixel, and geometry shader

with practically unlimited shader program length• Vertex, pixel, and geometry shader texture fetch

capabilities• Configurable output merger (Blending/Ops/Tests)• Fixed function pipeline vertex and/or pixel shader can

NOT be used anymore• Floating point through the entire pipeline including

blending• API Support for double precision (first hardware

supporting double expected to ship in late 2007)



“Basic HLSL” D3D 10/--------------------------------------------------------------------------------------// This shader outputs the pixel's color by modulating the texture's// color with diffuse material color//--------------------------------------------------------------------------------------PS_OUTPUT RenderScenePS( VS_OUTPUT In, uniform bool bTexture ) { PS_OUTPUT Output;

// Lookup mesh texture and modulate it with diffuse if( bTexture ) Output.RGBColor = g_MeshTexture.Sample(MeshTextureSampler, In.TextureUV) * In.Diffuse; else Output.RGBColor = In.Diffuse;

return Output;}

//--------------------------------------------------------------------------------------// Renders scene to render target using D3D10 Techniques//--------------------------------------------------------------------------------------technique10 RenderSceneWithTexture1Light{ pass P0 { SetVertexShader( CompileShader( vs_4_0, RenderSceneVS( 1, true, true ) ) ); SetGeometryShader( NULL ); SetPixelShader( CompileShader( ps_4_0, RenderScenePS( true ) ) );

SetDepthStencilState( EnableDepth, 0 ); }}



Coming up

GPU Effects

Computer graphics & visualization The programmable (D3D 10) Pipeline.

Documents

Transcript of Computer graphics & visualization The programmable (D3D 10) Pipeline.