State of the Art in Global Illumination for Interactive Applications and High-Quality Animations

INFORMATIKINFORMATIK

State of the Art in Global Illumination State of the Art in Global Illumination for Interactive Applications and for Interactive Applications and

High-Quality AnimationsHigh-Quality Animations

C. Damez, K. Dmitriev, and K. MyszkowskiC. Damez, K. Dmitriev, and K. Myszkowski

Max-Planck-Institut fMax-Planck-Institut für Informatikür Informatik

C. Damez, K. Dmitriev, and K. MyszkowskiC. Damez, K. Dmitriev, and K. Myszkowski

Max-Planck-Institut fMax-Planck-Institut für Informatikür Informatik

INFORMATIKINFORMATIKMotivationMotivation

• Global illumination in dynamic environments Global illumination in dynamic environments

– Important in many practical applications Important in many practical applications

– Increasing CPU and GPU power opens new Increasing CPU and GPU power opens new prospects for this fieldprospects for this field

• Algorithms, designed for dynamic environmentsAlgorithms, designed for dynamic environments

– Exploit temporal coherence:Exploit temporal coherence:

avoid redundant computationavoid redundant computation

remove temporal aliasingremove temporal aliasing

– Error metrics, which consider temporal aspectsError metrics, which consider temporal aspects

• Global illumination in dynamic environments Global illumination in dynamic environments

– Important in many practical applications Important in many practical applications

– Increasing CPU and GPU power opens new Increasing CPU and GPU power opens new prospects for this fieldprospects for this field

• Algorithms, designed for dynamic environmentsAlgorithms, designed for dynamic environments

– Exploit temporal coherence:Exploit temporal coherence:

avoid redundant computationavoid redundant computation

remove temporal aliasingremove temporal aliasing

– Error metrics, which consider temporal aspectsError metrics, which consider temporal aspects

INFORMATIKINFORMATIKGoalGoal

• Classify global illumination techniques in animated Classify global illumination techniques in animated scenesscenes

• Recall and discuss the most up-to-date developments Recall and discuss the most up-to-date developments in this fieldin this field

• Evaluate respective performances, pros and cons of Evaluate respective performances, pros and cons of the new developmentsthe new developments

FocusFocus• Techniques supporting dynamic environmentsTechniques supporting dynamic environments

in particular geometry changes are important for usin particular geometry changes are important for us

• General purpose lighting simulation algorithmsGeneral purpose lighting simulation algorithms

• Classify global illumination techniques in animated Classify global illumination techniques in animated scenesscenes

• Recall and discuss the most up-to-date developments Recall and discuss the most up-to-date developments in this fieldin this field

• Evaluate respective performances, pros and cons of Evaluate respective performances, pros and cons of the new developmentsthe new developments

FocusFocus• Techniques supporting dynamic environmentsTechniques supporting dynamic environments

in particular geometry changes are important for usin particular geometry changes are important for us

• General purpose lighting simulation algorithmsGeneral purpose lighting simulation algorithms

INFORMATIKINFORMATIKOverviewOverview

• Interactive methodsInteractive methods

– Radiosity (software and hardware solutions)Radiosity (software and hardware solutions)

– Stochastic techniquesStochastic techniques

– Hybrid techniquesHybrid techniques

– Render and shading cacheRender and shading cache

– Hardware-supported high quality display methodsHardware-supported high quality display methods

• Off-line methodsOff-line methods

– RadiosityRadiosity


– Perception guided techniquesPerception guided techniques

– High quality rendering methods – final gather techniquesHigh quality rendering methods – final gather techniques













Global Illumination for Global Illumination for Interactive EnvironmentsInteractive Environments

• Interactive use (e.g. design):Interactive use (e.g. design):

– Movement of objects isn’t known a prioriMovement of objects isn’t known a priori

– Geometric and material properties can changeGeometric and material properties can change

• Aiming at fast feedback to user change:Aiming at fast feedback to user change:

– A minimum frame rate should be ensuredA minimum frame rate should be ensured

– Image quality/precision can be traded for faster Image quality/precision can be traded for faster response timeresponse time

• Interactive use (e.g. design):Interactive use (e.g. design):

– Movement of objects isn’t known a prioriMovement of objects isn’t known a priori

– Geometric and material properties can changeGeometric and material properties can change

• Aiming at fast feedback to user change:Aiming at fast feedback to user change:

– A minimum frame rate should be ensuredA minimum frame rate should be ensured

– Image quality/precision can be traded for faster Image quality/precision can be traded for faster response timeresponse time

INFORMATIKINFORMATIKInstant Radiosity [Keller’97]Instant Radiosity [Keller’97]

• Coarse diffuse radiance approximation is received via photon shooting

• Each photon hit is treated as a point light• OpenGL rendering with shadows is used to sum up OpenGL rendering with shadows is used to sum up

contributions from all lightscontributions from all lights

• Coarse diffuse radiance approximation is received via photon shooting

• Each photon hit is treated as a point light• OpenGL rendering with shadows is used to sum up OpenGL rendering with shadows is used to sum up

contributions from all lightscontributions from all lights

• Walkthrough is possible by subtracting outdated contributions and adding new ones

• Walkthrough is possible by subtracting outdated contributions and adding new ones

Images : Keller Images : Keller et al.et al.Images : Keller Images : Keller et al.et al.

INFORMATIKINFORMATIKDiscussionDiscussion

• Relies exclusively on graphics hardware to Relies exclusively on graphics hardware to compute indirect illuminationcompute indirect illumination

fastfast

• View dependentView dependent

camera motion causes recomputationcamera motion causes recomputation

• Recomputation for dynamic scenes is not guided by importance

• Use of graphics hardware restricts the number of supported effects

• Relies exclusively on graphics hardware to Relies exclusively on graphics hardware to compute indirect illuminationcompute indirect illumination

fastfast

• View dependentView dependent

camera motion causes recomputationcamera motion causes recomputation

• Recomputation for dynamic scenes is not guided by importance


INFORMATIKINFORMATIKHybrid Rendering [Udeshi’99]Hybrid Rendering [Udeshi’99]

• Hardware is used to compute Hardware is used to compute shadows from point and area shadows from point and area lightslights

• One-bounce diffuse interreflections are approximated by virtual lights

• CPU computes reflections & refractions

• Hardware is used to compute Hardware is used to compute shadows from point and area shadows from point and area lightslights

• One-bounce diffuse interreflections are approximated by virtual lights

• CPU computes reflections & refractions

• 1-10 fps on computer with several graphics pipelines1-10 fps on computer with several graphics pipelines• 1-10 fps on computer with several graphics pipelines1-10 fps on computer with several graphics pipelines

Images : Udeshi Images : Udeshi et al.et al.Images : Udeshi Images : Udeshi et al.et al.


Selective Photon Tracing Selective Photon Tracing [Dmitriev’02][Dmitriev’02]

• Indirect illumination is computed for each vertex via photon shooting

• Computation is prioritized by perceptual importance of Computation is prioritized by perceptual importance of illumination updateillumination update

• Direct illumination is computed by graphics hardware, Direct illumination is computed by graphics hardware, including shadows and goniometric diagramsincluding shadows and goniometric diagrams

• Indirect illumination is computed for each vertex via photon shooting

• Computation is prioritized by perceptual importance of Computation is prioritized by perceptual importance of illumination updateillumination update

• Direct illumination is computed by graphics hardware, Direct illumination is computed by graphics hardware, including shadows and goniometric diagramsincluding shadows and goniometric diagrams

INFORMATIKINFORMATIKAlgorithm Strategy (1)Algorithm Strategy (1)

Initial solution is received using a fixed pool of photon paths

Initial solution is received using a fixed pool of photon paths

Scene changes perturb some of

those paths, which cause changes in

illumination

Scene changes perturb some of

those paths, which cause changes in

illumination


• Find important paths by sparsely sampling the space with pilot photons

• Select similar paths from the photon pool, using Select similar paths from the photon pool, using periodicity of the Halton sequenceperiodicity of the Halton sequence

• Find important paths by sparsely sampling the space with pilot photons

• Select similar paths from the photon pool, using Select similar paths from the photon pool, using periodicity of the Halton sequenceperiodicity of the Halton sequence

kjgk

jjmNiij lbNb

hhg

if ,1

|| )(


• Frame rate 1-8 fps, but full illumination recovery can take up to 10 sec

• View independentView independent

camera motion doesn’t cause recomputation

• No adaptive geometry subdivisionNo adaptive geometry subdivision

lighting artifacts (e.g. leaks) are possible


• Frame rate 1-8 fps, but full illumination recovery can take up to 10 sec

• View independentView independent

camera motion doesn’t cause recomputation

• No adaptive geometry subdivisionNo adaptive geometry subdivision

lighting artifacts (e.g. leaks) are possible



Distributed Ray Tracing API Distributed Ray Tracing API [Wald’02][Wald’02]

• Instant Radiosity approach is extended by using fast ray tracing instead of graphics hardware

• Caustic photons are shot to simulate causticsCaustic photons are shot to simulate caustics

• Computation is distributed over a cluster of Computation is distributed over a cluster of PCs, interleaving images from different clientsPCs, interleaving images from different clients

• Discontinuity buffer heuristic is used to filter out the Monte Carlo noise

• Instant Radiosity approach is extended by using fast ray tracing instead of graphics hardware

• Caustic photons are shot to simulate causticsCaustic photons are shot to simulate caustics

• Computation is distributed over a cluster of Computation is distributed over a cluster of PCs, interleaving images from different clientsPCs, interleaving images from different clients

• Discontinuity buffer heuristic is used to filter out the Monte Carlo noise

Images : Wald Images : Wald et al.et al.Images : Wald Images : Wald et al.et al.


• Each PC is assigned a pixel from the tile

• Each PC uses own Each PC uses own radiance radiance representation and own representation and own set of caustic photonsset of caustic photons

• Each PC is assigned a pixel from the tile

• Each PC uses own Each PC uses own radiance radiance representation and own representation and own set of caustic photonsset of caustic photons

Image

Image tile


• Heuristic, based on normals and distances, is used to filter irradiance values from different clients

• Indirect diffuse, caustic and direct illumination is involved into the filtering process

• Heuristic, based on normals and distances, is used to filter irradiance values from different clients

• Indirect diffuse, caustic and direct illumination is involved into the filtering process

Images : Wald Images : Wald et al.et al.Images : Wald Images : Wald et al.et al.


• Frame rate 1-10 fps, but computing final image can Frame rate 1-10 fps, but computing final image can take up to 2 secondstake up to 2 seconds

• Wide range of effects is supported (reflections, Wide range of effects is supported (reflections, refractions, caustics)refractions, caustics)

• There is no outdated illuminationThere is no outdated illumination

• Involves huge computational power (PC cluster is Involves huge computational power (PC cluster is required)required)

• Image flickering is observed until enough samples Image flickering is observed until enough samples are accumulated after scene or camera changeare accumulated after scene or camera change

• Frame rate 1-10 fps, but computing final image can Frame rate 1-10 fps, but computing final image can take up to 2 secondstake up to 2 seconds

• Wide range of effects is supported (reflections, Wide range of effects is supported (reflections, refractions, caustics)refractions, caustics)

• There is no outdated illuminationThere is no outdated illumination

• Involves huge computational power (PC cluster is Involves huge computational power (PC cluster is required)required)

• Image flickering is observed until enough samples Image flickering is observed until enough samples are accumulated after scene or camera changeare accumulated after scene or camera change


Incremental Update of Incremental Update of RadiosityRadiosity

• First attempts at interactive global illumination based First attempts at interactive global illumination based on finite element methods :on finite element methods :

– Progressive Radiosity [Chen’90,George’90]Progressive Radiosity [Chen’90,George’90]

– Hierarchical Radiosity [Forsyth’94,Shaw’97]Hierarchical Radiosity [Forsyth’94,Shaw’97]

• Limited to very simple scenesLimited to very simple scenes

• Quality/time tradeoff difficult to tuneQuality/time tradeoff difficult to tune

• First attempts at interactive global illumination based First attempts at interactive global illumination based on finite element methods :on finite element methods :

– Progressive Radiosity [Chen’90,George’90]Progressive Radiosity [Chen’90,George’90]

– Hierarchical Radiosity [Forsyth’94,Shaw’97]Hierarchical Radiosity [Forsyth’94,Shaw’97]

• Limited to very simple scenesLimited to very simple scenes

• Quality/time tradeoff difficult to tuneQuality/time tradeoff difficult to tune


Line-Space Hierarchy Line-Space Hierarchy [Drettakis’97][Drettakis’97]

• Based on Hierarchical Radiosity with ClusteringBased on Hierarchical Radiosity with Clustering

• Starts from a static solutionStarts from a static solution

• Augments HR link structure with shaftsAugments HR link structure with shafts

• Hierarchical test dynamic object vs shaftsHierarchical test dynamic object vs shafts

• Based on Hierarchical Radiosity with ClusteringBased on Hierarchical Radiosity with Clustering

• Starts from a static solutionStarts from a static solution

• Augments HR link structure with shaftsAugments HR link structure with shafts

• Hierarchical test dynamic object vs shaftsHierarchical test dynamic object vs shafts

pp qq


• Time/quality tradeoff :Time/quality tradeoff :

– max # links to refinemax # links to refine

– max # links to updatemax # links to update

• Always computes a consistent Always computes a consistent solutionsolution

• Memory intensive, but the Memory intensive, but the storage of shafts can be storage of shafts can be avoided [Schoeffel’99]avoided [Schoeffel’99]

• Time/quality tradeoff :Time/quality tradeoff :

– max # links to refinemax # links to refine

– max # links to updatemax # links to update

• Always computes a consistent Always computes a consistent solutionsolution

• Memory intensive, but the Memory intensive, but the storage of shafts can be storage of shafts can be avoided [Schoeffel’99]avoided [Schoeffel’99]

Images : Drettakis Images : Drettakis et al.et al.Images : Drettakis Images : Drettakis et al.et al.


Unified Hierarchical Unified Hierarchical Algorithm [Granier’00]Algorithm [Granier’00]

• Global illumination with glossy surfaces:Global illumination with glossy surfaces:

– Diffuse lighting: Hierarchical RadiosityDiffuse lighting: Hierarchical Radiosity

– Glossy effects: Particle tracing guided by the link Glossy effects: Particle tracing guided by the link hierarchyhierarchy

• Shafts bound particles transferShafts bound particles transfer

• Particles are stored in mesh or in texturesParticles are stored in mesh or in textures

• Specular paths to the eye are computed as a post Specular paths to the eye are computed as a post processprocess

• Global illumination with glossy surfaces:Global illumination with glossy surfaces:

– Diffuse lighting: Hierarchical RadiosityDiffuse lighting: Hierarchical Radiosity

– Glossy effects: Particle tracing guided by the link Glossy effects: Particle tracing guided by the link hierarchyhierarchy

• Shafts bound particles transferShafts bound particles transfer

• Particles are stored in mesh or in texturesParticles are stored in mesh or in textures

• Specular paths to the eye are computed as a post Specular paths to the eye are computed as a post processprocess

INFORMATIKINFORMATIKInteraction with the UHAInteraction with the UHA

• Framework similar to the Line-Space Framework similar to the Line-Space HierarchyHierarchy

• Identify particles paths affected by Identify particles paths affected by motion: motion: – Construction of an octree Construction of an octree

subdivided around the dynamic subdivided around the dynamic object positionobject position

– Links are inserted in all cells they Links are inserted in all cells they traversetraverse

• # particles resent during motion is # particles resent during motion is limited by the userlimited by the user

• Framework similar to the Line-Space Framework similar to the Line-Space HierarchyHierarchy

• Identify particles paths affected by Identify particles paths affected by motion: motion: – Construction of an octree Construction of an octree

subdivided around the dynamic subdivided around the dynamic object positionobject position

– Links are inserted in all cells they Links are inserted in all cells they traversetraverse

• # particles resent during motion is # particles resent during motion is limited by the userlimited by the user

Images : Granier Images : Granier et al.et al.Images : Granier Images : Granier et al.et al.


• Allows quasi-interactive updates of Allows quasi-interactive updates of global illumination solutions (~0.5 fps)global illumination solutions (~0.5 fps)

• Degraded quality during interaction to Degraded quality during interaction to maintain a reasonable frameratemaintain a reasonable framerate

• Requires a static solution to be Requires a static solution to be computed beforehandscomputed beforehands

• Allows quasi-interactive updates of Allows quasi-interactive updates of global illumination solutions (~0.5 fps)global illumination solutions (~0.5 fps)

• Degraded quality during interaction to Degraded quality during interaction to maintain a reasonable frameratemaintain a reasonable framerate

• Requires a static solution to be Requires a static solution to be computed beforehandscomputed beforehands

Images : Granier Images : Granier et al.et al.Images : Granier Images : Granier et al.et al.


Render-Cache : PrinciplesRender-Cache : Principles[Walter’99][Walter’99]

• Decouples global illumination computations from Decouples global illumination computations from image rendering (frameless rendering)image rendering (frameless rendering)

• Reconstructs the illumination from a sparse set of Reconstructs the illumination from a sparse set of samples in image spacesamples in image space

• Purely software approach based on point Purely software approach based on point reprojection and adaptive samplingreprojection and adaptive sampling

• Decouples global illumination computations from Decouples global illumination computations from image rendering (frameless rendering)image rendering (frameless rendering)

• Reconstructs the illumination from a sparse set of Reconstructs the illumination from a sparse set of samples in image spacesamples in image space

• Purely software approach based on point Purely software approach based on point reprojection and adaptive samplingreprojection and adaptive sampling

INFORMATIKINFORMATIKImage ReconstructionImage Reconstruction

• Points in cache are reprojected on screenPoints in cache are reprojected on screen

• After reprojection, occluded points are removed by a After reprojection, occluded points are removed by a depth-culling heuristicdepth-culling heuristic

• Holes filled by interpolation and filteringHoles filled by interpolation and filtering

• Points in cache are reprojected on screenPoints in cache are reprojected on screen

• After reprojection, occluded points are removed by a After reprojection, occluded points are removed by a depth-culling heuristicdepth-culling heuristic

• Holes filled by interpolation and filteringHoles filled by interpolation and filtering

Images : Walter Images : Walter et al.et al.Images : Walter Images : Walter et al.et al.

INFORMATIKINFORMATIKChoice of SamplesChoice of Samples

• New samples are computed asynchronouslyNew samples are computed asynchronously• A grayscale priority image indicates regions most A grayscale priority image indicates regions most needing sampleneeding sample• Dithering is used to obtain sample positionsDithering is used to obtain sample positions

• New samples are computed asynchronouslyNew samples are computed asynchronously• A grayscale priority image indicates regions most A grayscale priority image indicates regions most needing sampleneeding sample• Dithering is used to obtain sample positionsDithering is used to obtain sample positions

Images : Walter Images : Walter et al.et al.Images : Walter Images : Walter et al.et al.


• Designed for ray tracing but can also be used with Designed for ray tracing but can also be used with path tracerspath tracers

• Can be used to display specular surfaces in finite Can be used to display specular surfaces in finite element solutions (e.g. UHA)element solutions (e.g. UHA)

• Scalable with high image resolution if carefully Scalable with high image resolution if carefully implemented [Walter’02]implemented [Walter’02]

• Rapid movements may cause distracting artifactsRapid movements may cause distracting artifacts

• Designed for ray tracing but can also be used with Designed for ray tracing but can also be used with path tracerspath tracers

• Can be used to display specular surfaces in finite Can be used to display specular surfaces in finite element solutions (e.g. UHA)element solutions (e.g. UHA)

• Scalable with high image resolution if carefully Scalable with high image resolution if carefully implemented [Walter’02]implemented [Walter’02]

• Rapid movements may cause distracting artifactsRapid movements may cause distracting artifacts


Shading-Cache: PrinciplesShading-Cache: Principles[Tole’02][Tole’02]

• Perform reconstruction in 3D spacePerform reconstruction in 3D space

avoids holes and occlusion errorsavoids holes and occlusion errors

• Cached samples are used to update and refine a Cached samples are used to update and refine a 3D mesh progressively3D mesh progressively

• Display using the graphic pipelineDisplay using the graphic pipeline

high frameratehigh framerate

correct geometric representation of objects correct geometric representation of objects

temporary shading inaccuraciestemporary shading inaccuracies

• Perform reconstruction in 3D spacePerform reconstruction in 3D space

avoids holes and occlusion errorsavoids holes and occlusion errors

• Cached samples are used to update and refine a Cached samples are used to update and refine a 3D mesh progressively3D mesh progressively

• Display using the graphic pipelineDisplay using the graphic pipeline

high frameratehigh framerate

correct geometric representation of objects correct geometric representation of objects

temporary shading inaccuraciestemporary shading inaccuracies

INFORMATIKINFORMATIKSampling & Mesh RefinementSampling & Mesh Refinement

• Priority image is built using difference in color Priority image is built using difference in color between vertices of a given patchbetween vertices of a given patch

used as used as pdfpdf to randomly select patches for update to randomly select patches for update or refinementor refinement

• To limit the size of the mesh, patches moving out of To limit the size of the mesh, patches moving out of the the fov fov are removedare removed

• Priority image is built using difference in color Priority image is built using difference in color between vertices of a given patchbetween vertices of a given patch

used as used as pdfpdf to randomly select patches for update to randomly select patches for update or refinementor refinement

• To limit the size of the mesh, patches moving out of To limit the size of the mesh, patches moving out of the the fov fov are removedare removed

Images : Tole Images : Tole et al.et al.Images : Tole Images : Tole et al.et al.


• Requires ~10x less samples than Requires ~10x less samples than the Render Cache to build an the Render Cache to build an imageimage

much faster updates with a much faster updates with a global illumination samplerglobal illumination sampler

• Framerate ~40-60 fps but shading Framerate ~40-60 fps but shading update may require ~10supdate may require ~10s

manipulating objects easier and manipulating objects easier and more intuitivemore intuitive

• Requires ~10x less samples than Requires ~10x less samples than the Render Cache to build an the Render Cache to build an imageimage

much faster updates with a much faster updates with a global illumination samplerglobal illumination sampler

• Framerate ~40-60 fps but shading Framerate ~40-60 fps but shading update may require ~10supdate may require ~10s

manipulating objects easier and manipulating objects easier and more intuitivemore intuitive

Images : Tole Images : Tole et al.et al.Images : Tole Images : Tole et al.et al.


Hardware Supported Display Hardware Supported Display of Specular Effectsof Specular Effects• Mirror reflections for planar surfaces Mirror reflections for planar surfaces

[Diefenbach’94, Wojdala’94][Diefenbach’94, Wojdala’94]

– Separate rendering pass for each mirrorSeparate rendering pass for each mirror

– Recursion of the basic algorithm for the Recursion of the basic algorithm for the higher order reflectionshigher order reflections

• Refractions [Diefenbach’94]Refractions [Diefenbach’94]

– Snell’s law: non-linear transformation Snell’s law: non-linear transformation difficult to implement in hardwaredifficult to implement in hardware

– Approximation by the tangent lawApproximation by the tangent law

• Reflections for curved surfaces: environment Reflections for curved surfaces: environment maps [Green’86, Haeberli’93, Voorhies’94]maps [Green’86, Haeberli’93, Voorhies’94]

– Approximated reflectionsApproximated reflections

– No motion parallax No motion parallax

• Mirror reflections for planar surfaces Mirror reflections for planar surfaces [Diefenbach’94, Wojdala’94][Diefenbach’94, Wojdala’94]

– Separate rendering pass for each mirrorSeparate rendering pass for each mirror

– Recursion of the basic algorithm for the Recursion of the basic algorithm for the higher order reflectionshigher order reflections

• Refractions [Diefenbach’94]Refractions [Diefenbach’94]

– Snell’s law: non-linear transformation Snell’s law: non-linear transformation difficult to implement in hardwaredifficult to implement in hardware

– Approximation by the tangent lawApproximation by the tangent law

• Reflections for curved surfaces: environment Reflections for curved surfaces: environment maps [Green’86, Haeberli’93, Voorhies’94]maps [Green’86, Haeberli’93, Voorhies’94]

– Approximated reflectionsApproximated reflections

– No motion parallax No motion parallax


Hardware Supported Display Hardware Supported Display of Glossy Effectsof Glossy Effects• Pre-filtered environment mapsPre-filtered environment maps: For a given : For a given

reflectance function store lighting reflected reflectance function store lighting reflected from the whole environment for all viewing from the whole environment for all viewing directionsdirections

– Diffuse reflectance [Greene’86]Diffuse reflectance [Greene’86]

– Phong reflectance (with a Fresnel term for Phong reflectance (with a Fresnel term for plastic appearance) [Heidrich’99]plastic appearance) [Heidrich’99]

– Arbitrary BRDFs with rotationally symmetric Arbitrary BRDFs with rotationally symmetric lobes [Kautz’00]lobes [Kautz’00]

• Homomorphic Factorization: Homomorphic Factorization: separation of separation of arbitrary BRDFs into 2D texturesarbitrary BRDFs into 2D textures

– point and directional light sources point and directional light sources [McCool’01][McCool’01]

– full global illumination [Latta’02] (isotropic full global illumination [Latta’02] (isotropic BRDFs only)BRDFs only)

• Pre-filtered environment mapsPre-filtered environment maps: For a given : For a given reflectance function store lighting reflected reflectance function store lighting reflected from the whole environment for all viewing from the whole environment for all viewing directionsdirections

– Diffuse reflectance [Greene’86]Diffuse reflectance [Greene’86]

– Phong reflectance (with a Fresnel term for Phong reflectance (with a Fresnel term for plastic appearance) [Heidrich’99]plastic appearance) [Heidrich’99]

– Arbitrary BRDFs with rotationally symmetric Arbitrary BRDFs with rotationally symmetric lobes [Kautz’00]lobes [Kautz’00]

• Homomorphic Factorization: Homomorphic Factorization: separation of separation of arbitrary BRDFs into 2D texturesarbitrary BRDFs into 2D textures

– point and directional light sources point and directional light sources [McCool’01][McCool’01]

– full global illumination [Latta’02] (isotropic full global illumination [Latta’02] (isotropic BRDFs only)BRDFs only) Images: Kautz et al.


Efficient Prefiltering of Efficient Prefiltering of Environment MapsEnvironment Maps

Problem:Problem: Dynamic scenes require frequent Dynamic scenes require frequent update of prefiltered environement maps update of prefiltered environement maps • Hierarchical prefiltering [Kautz’00] – softwareHierarchical prefiltering [Kautz’00] – software• Hardware accelerated prefiltering [Kautz’00]Hardware accelerated prefiltering [Kautz’00]

– Support for shift-invariant filters of only limited sizeSupport for shift-invariant filters of only limited size– Phong reflectance can be processed but the map Phong reflectance can be processed but the map

must be shrunk to match the filter sizemust be shrunk to match the filter size• Spherical frequency space analysis [Ramamoorthi’01a]Spherical frequency space analysis [Ramamoorthi’01a]

– Irradiance environment maps - Lambertian-like Irradiance environment maps - Lambertian-like surfaces [Ramamoorthi’01b]surfaces [Ramamoorthi’01b]

– Spherical harmonics reflection maps – surfaces with Spherical harmonics reflection maps – surfaces with complex isotropic BRDFs [Ramamoorthi’02]complex isotropic BRDFs [Ramamoorthi’02]

Problem:Problem: Dynamic scenes require frequent Dynamic scenes require frequent update of prefiltered environement maps update of prefiltered environement maps • Hierarchical prefiltering [Kautz’00] – softwareHierarchical prefiltering [Kautz’00] – software• Hardware accelerated prefiltering [Kautz’00]Hardware accelerated prefiltering [Kautz’00]

– Support for shift-invariant filters of only limited sizeSupport for shift-invariant filters of only limited size– Phong reflectance can be processed but the map Phong reflectance can be processed but the map

must be shrunk to match the filter sizemust be shrunk to match the filter size• Spherical frequency space analysis [Ramamoorthi’01a]Spherical frequency space analysis [Ramamoorthi’01a]

– Irradiance environment maps - Lambertian-like Irradiance environment maps - Lambertian-like surfaces [Ramamoorthi’01b]surfaces [Ramamoorthi’01b]

– Spherical harmonics reflection maps – surfaces with Spherical harmonics reflection maps – surfaces with complex isotropic BRDFs [Ramamoorthi’02]complex isotropic BRDFs [Ramamoorthi’02]


Using Environment Mapping for Using Environment Mapping for Interactive Global IlluminationInteractive Global Illumination

Irradiance Volumes [Greger’98]Irradiance Volumes [Greger’98]

revisited [Mantiuk]revisited [Mantiuk]• Divide 3D space into a grid of volumesDivide 3D space into a grid of volumes

• For each bounce of interreflectionFor each bounce of interreflection

1 For each node of the grid render a cube For each node of the grid render a cube map and compute spherical harmonic map and compute spherical harmonic coefficients representing its irradiance coefficients representing its irradiance [Ramamoorthi’01][Ramamoorthi’01]

2 For each vertex of any object interactively For each vertex of any object interactively manipulated by the user interpolate its manipulated by the user interpolate its irradiance based on the neighboring nodes irradiance based on the neighboring nodes

3 Asynchronously render and update Asynchronously render and update irradiance at the nodesirradiance at the nodes

Irradiance Volumes [Greger’98]Irradiance Volumes [Greger’98]

revisited [Mantiuk]revisited [Mantiuk]• Divide 3D space into a grid of volumesDivide 3D space into a grid of volumes

• For each bounce of interreflectionFor each bounce of interreflection

1 For each node of the grid render a cube For each node of the grid render a cube map and compute spherical harmonic map and compute spherical harmonic coefficients representing its irradiance coefficients representing its irradiance [Ramamoorthi’01][Ramamoorthi’01]

2 For each vertex of any object interactively For each vertex of any object interactively manipulated by the user interpolate its manipulated by the user interpolate its irradiance based on the neighboring nodes irradiance based on the neighboring nodes

3 Asynchronously render and update Asynchronously render and update irradiance at the nodesirradiance at the nodes

Images: Mantiuk et al.


Precomputed Radiance Precomputed Radiance Transfer Function [Sloan’02]Transfer Function [Sloan’02]• Real-time rendering in dynamic, low-frequency lighting Real-time rendering in dynamic, low-frequency lighting

environments environments

– Self-shadows and self-interreflection effectsSelf-shadows and self-interreflection effects

– Possible extensions to neighboring objects: caustics and soft Possible extensions to neighboring objects: caustics and soft shadowsshadows

• Basic algorithm Basic algorithm

– Preprocessing: Computing radiance transfer function (RTF) for Preprocessing: Computing radiance transfer function (RTF) for each rigid objecteach rigid object

– At run-time for each object At run-time for each object Projecting incident radiance to spherical harmonics and Projecting incident radiance to spherical harmonics and

applying to the RTFapplying to the RTF Convolving self-scattered radiance with the object’s BRDF Convolving self-scattered radiance with the object’s BRDF

and evaluating at the view-dependent reflection directionand evaluating at the view-dependent reflection direction

– Precomputed Precomputed

– Efficiently handles object self-shadows and self-interreflection Efficiently handles object self-shadows and self-interreflection effectseffects

• Real-time rendering in dynamic, low-frequency lighting Real-time rendering in dynamic, low-frequency lighting environments environments

– Self-shadows and self-interreflection effectsSelf-shadows and self-interreflection effects

– Possible extensions to neighboring objects: caustics and soft Possible extensions to neighboring objects: caustics and soft shadowsshadows

• Basic algorithm Basic algorithm

– Preprocessing: Computing radiance transfer function (RTF) for Preprocessing: Computing radiance transfer function (RTF) for each rigid objecteach rigid object

– At run-time for each object At run-time for each object Projecting incident radiance to spherical harmonics and Projecting incident radiance to spherical harmonics and

applying to the RTFapplying to the RTF Convolving self-scattered radiance with the object’s BRDF Convolving self-scattered radiance with the object’s BRDF

and evaluating at the view-dependent reflection directionand evaluating at the view-dependent reflection direction

– Precomputed Precomputed

– Efficiently handles object self-shadows and self-interreflection Efficiently handles object self-shadows and self-interreflection effectseffects Images: Sloan et al.

INFORMATIKINFORMATIKOverviewOverview
























Global Illumination in Global Illumination in AnimationsAnimations

• Aiming at higher quality animations:Aiming at higher quality animations:

– Constant image quality should be ensuredConstant image quality should be ensured

– Popping artifacts should be avoidedPopping artifacts should be avoided

• Computing time is not constrained Computing time is not constrained

• Benefit from trajectories knowledgeBenefit from trajectories knowledge

• Aiming at higher quality animations:Aiming at higher quality animations:

– Constant image quality should be ensuredConstant image quality should be ensured

– Popping artifacts should be avoidedPopping artifacts should be avoided

• Computing time is not constrained Computing time is not constrained

• Benefit from trajectories knowledgeBenefit from trajectories knowledge


Space-Time Hierarchical Space-Time Hierarchical Radiosity [Damez’99] (1)Radiosity [Damez’99] (1)

• Aims at computing diffuse global illumination solutions Aims at computing diffuse global illumination solutions for animationsfor animations

• Based on Hierarchical RadiosityBased on Hierarchical Radiosity

• Radiosity is a function of space & timeRadiosity is a function of space & time

• New radiosity equation:New radiosity equation:

• Formally equivalent to the static equationFormally equivalent to the static equation

• Aims at computing diffuse global illumination solutions Aims at computing diffuse global illumination solutions for animationsfor animations

• Based on Hierarchical RadiosityBased on Hierarchical Radiosity

• Radiosity is a function of space & timeRadiosity is a function of space & time

• New radiosity equation:New radiosity equation:

• Formally equivalent to the static equationFormally equivalent to the static equation

',

,)(,,tqY

dYYXKYBXXEXBtpX

',

,)(,,tqY

dYYXKYBXXEXBtpX

',,,,,',,, tttqpvtqpgtqtpK ',,,,,',,, tttqpvtqpgtqtpK


Space-Time Hierarchical Space-Time Hierarchical Radiosity [Damez’99] (2)Radiosity [Damez’99] (2)

• Hierarchical solution : space-time meshHierarchical solution : space-time mesh– Mesh element is polygon x time Mesh element is polygon x time

intervalinterval– Space or time subdivision Space or time subdivision

• Can also be applied:Can also be applied:– ClusteringClustering– Wavelet-basis for the time dimensionWavelet-basis for the time dimension

• Hierarchical solution : space-time meshHierarchical solution : space-time mesh– Mesh element is polygon x time Mesh element is polygon x time

intervalinterval– Space or time subdivision Space or time subdivision

• Can also be applied:Can also be applied:– ClusteringClustering– Wavelet-basis for the time dimensionWavelet-basis for the time dimension

tt11

tt00

tt11

tt00 tt11

tt00

tt22


• Refinement is the key issueRefinement is the key issue

• Memory consumption is highMemory consumption is high

• Performs better on “long” sequences and complex Performs better on “long” sequences and complex scenes (speedup = 7x-12x)scenes (speedup = 7x-12x)

• Refinement is the key issueRefinement is the key issue

• Memory consumption is highMemory consumption is high

• Performs better on “long” sequences and complex Performs better on “long” sequences and complex scenes (speedup = 7x-12x)scenes (speedup = 7x-12x)


Multi-Frame Lighting Method Multi-Frame Lighting Method [Besuievsky’96][Besuievsky’96]

• Global Monte Carlo method for Global Monte Carlo method for animated scenesanimated scenes

• Based on global multipath Monte Based on global multipath Monte Carlo algorithm [Sbert’96]Carlo algorithm [Sbert’96]

• All positions of moving objects are All positions of moving objects are inserted in the sceneinserted in the scene

• Use the same set of global lines Use the same set of global lines for the whole animationfor the whole animation

• Global Monte Carlo method for Global Monte Carlo method for animated scenesanimated scenes

• Based on global multipath Monte Based on global multipath Monte Carlo algorithm [Sbert’96]Carlo algorithm [Sbert’96]

• All positions of moving objects are All positions of moving objects are inserted in the sceneinserted in the scene

• Use the same set of global lines Use the same set of global lines for the whole animationfor the whole animation


• Reused global lines ensure Reused global lines ensure temporal coherencetemporal coherence

• Speedup = 7x-25xSpeedup = 7x-25x

• Adaptive meshing is not Adaptive meshing is not possible yetpossible yet

• Dense meshes increase noise Dense meshes increase noise and memory consumptionand memory consumption

no high frequency detailsno high frequency details

• Reused global lines ensure Reused global lines ensure temporal coherencetemporal coherence

• Speedup = 7x-25xSpeedup = 7x-25x

• Adaptive meshing is not Adaptive meshing is not possible yetpossible yet

• Dense meshes increase noise Dense meshes increase noise and memory consumptionand memory consumption

no high frequency detailsno high frequency detailsImages : Besuievesky Images : Besuievesky et al.et al.Images : Besuievesky Images : Besuievesky et al.et al.


Image-Based Global Illumination Image-Based Global Illumination Framework [Nimeroff’96]Framework [Nimeroff’96]

• ““View Space” spanned by the base range imagesView Space” spanned by the base range images

– Key camera positions inserted adaptively based on Key camera positions inserted adaptively based on changes in object visibility sampled by the item changes in object visibility sampled by the item bufferbuffer

– Visibility of reflected/refracted objects ignoredVisibility of reflected/refracted objects ignored

• Global illumination changes sparsely sampled in time Global illumination changes sparsely sampled in time

– Adaptive time steps for updating illumination Adaptive time steps for updating illumination

– Asynchronous processing of direct and indirect Asynchronous processing of direct and indirect lighting lighting

• Rendering through image warping and lighting Rendering through image warping and lighting interpolation between keyframes interpolation between keyframes

• ““View Space” spanned by the base range imagesView Space” spanned by the base range images

– Key camera positions inserted adaptively based on Key camera positions inserted adaptively based on changes in object visibility sampled by the item changes in object visibility sampled by the item bufferbuffer

– Visibility of reflected/refracted objects ignoredVisibility of reflected/refracted objects ignored

• Global illumination changes sparsely sampled in time Global illumination changes sparsely sampled in time

– Adaptive time steps for updating illumination Adaptive time steps for updating illumination

– Asynchronous processing of direct and indirect Asynchronous processing of direct and indirect lighting lighting

• Rendering through image warping and lighting Rendering through image warping and lighting interpolation between keyframes interpolation between keyframes


+ Free design of camera path within view spaceFree design of camera path within view spaceGood approximation of global illumination for the Good approximation of global illumination for the

whole dynamic sequencewhole dynamic sequence+ Reduced temporal aliasing due to illumination Reduced temporal aliasing due to illumination

interpolationinterpolation+ On current graphics hardware interactive On current graphics hardware interactive

implementation could be possibleimplementation could be possible– Tone mapping not includedTone mapping not included

But the problem could be solved with HDR imagesBut the problem could be solved with HDR images– Excessively conservative computationExcessively conservative computation

Direct and indirect lighting patterns can wash out Direct and indirect lighting patterns can wash out each other while are processed asynchronouslyeach other while are processed asynchronously

+ Free design of camera path within view spaceFree design of camera path within view spaceGood approximation of global illumination for the Good approximation of global illumination for the

whole dynamic sequencewhole dynamic sequence+ Reduced temporal aliasing due to illumination Reduced temporal aliasing due to illumination

interpolationinterpolation+ On current graphics hardware interactive On current graphics hardware interactive

implementation could be possibleimplementation could be possible– Tone mapping not includedTone mapping not included

But the problem could be solved with HDR imagesBut the problem could be solved with HDR images– Excessively conservative computationExcessively conservative computation

Direct and indirect lighting patterns can wash out Direct and indirect lighting patterns can wash out each other while are processed asynchronouslyeach other while are processed asynchronously


Perception Guided Animation Perception Guided Animation Rendering - PrinciplesRendering - Principles

• Goal: take into account characteristics of the Human Goal: take into account characteristics of the Human Visual System to concentrate the image computation Visual System to concentrate the image computation exclusively on the visible scene details.exclusively on the visible scene details.

• Perception-based computation guidance Perception-based computation guidance

– Early stages of the visual path well understood Early stages of the visual path well understood

Video and animation quality metrics (AQM) are Video and animation quality metrics (AQM) are availableavailable

– Higher level perceptual and cognitive models Higher level perceptual and cognitive models poorly elaboratedpoorly elaborated

Visual attention models available, but possibly Visual attention models available, but possibly not very robustnot very robust

• Goal: take into account characteristics of the Human Goal: take into account characteristics of the Human Visual System to concentrate the image computation Visual System to concentrate the image computation exclusively on the visible scene details.exclusively on the visible scene details.

• Perception-based computation guidance Perception-based computation guidance

– Early stages of the visual path well understood Early stages of the visual path well understood

Video and animation quality metrics (AQM) are Video and animation quality metrics (AQM) are availableavailable

– Higher level perceptual and cognitive models Higher level perceptual and cognitive models poorly elaboratedpoorly elaborated

Visual attention models available, but possibly Visual attention models available, but possibly not very robustnot very robust

INFORMATIKINFORMATIKAnimation Quality Metric (AQM)Animation Quality Metric (AQM)

• Computes the map of visible Computes the map of visible differences differences between two input animation framesbetween two input animation frames

• HVS modeling scope:HVS modeling scope:– Weber law Weber law – Spatio-velocity Contrast Sensitivity FunctionSpatio-velocity Contrast Sensitivity Function– Visual masking Visual masking

• Computes the map of visible Computes the map of visible differences differences between two input animation framesbetween two input animation frames

• HVS modeling scope:HVS modeling scope:– Weber law Weber law – Spatio-velocity Contrast Sensitivity FunctionSpatio-velocity Contrast Sensitivity Function– Visual masking Visual masking


Spatio-Temporal Density Spatio-Temporal Density Estimation Particle TracingEstimation Particle Tracing

• View-independent light source photon tracingView-independent light source photon tracing• View-independent light source photon tracingView-independent light source photon tracing


• View-independent light source photon tracing View-independent light source photon tracing • View-independent light source photon tracing View-independent light source photon tracing



• Extending photon processing into the temporal domain Extending photon processing into the temporal domain • Extending photon processing into the temporal domain Extending photon processing into the temporal domain



Temporal Photon Processing: Temporal Photon Processing: Contradictory RequirementContradictory Requirement

• Maximize the number of photons collected in the Maximize the number of photons collected in the temporal domain to reduce the stochastic noise.temporal domain to reduce the stochastic noise.

• Minimize the time interval in which the photons were Minimize the time interval in which the photons were traced to avoid collecting invalid photons.traced to avoid collecting invalid photons.

• Maximize the number of photons collected in the Maximize the number of photons collected in the temporal domain to reduce the stochastic noise.temporal domain to reduce the stochastic noise.

• Minimize the time interval in which the photons were Minimize the time interval in which the photons were traced to avoid collecting invalid photons.traced to avoid collecting invalid photons.

Static object

Moving object


Animation Animation Quality Quality MetricMetric

Are the differences Are the differences acceptable ?acceptable ?

YESYESNONO– Shoot more Shoot more photonsphotons

– RecurseRecurse

Generate Generate inbetween inbetween

imagesimages

Perception-Based Perception-Based Computation GuidanceComputation Guidance

odd photons even photons

• AQM: AQM: Decides upon the computation stopping conditionDecides upon the computation stopping condition

– ComputedComputed once per animation segment for a central once per animation segment for a central frame Kframe K

• AQM: AQM: Decides upon the computation stopping conditionDecides upon the computation stopping condition

– ComputedComputed once per animation segment for a central once per animation segment for a central frame Kframe K


+ Significantly reduced the number of Significantly reduced the number of photons to be shot per framephotons to be shot per frame

+ Drastically reduced temporal aliasingDrastically reduced temporal aliasing

– Limited spatial resolution of mesh Limited spatial resolution of mesh reconstructed lightingreconstructed lighting

– For quickly changing indirect lighting For quickly changing indirect lighting temporal processing can be limitedtemporal processing can be limited

Spatial filtering can be performed at Spatial filtering can be performed at the expense of loosing spatial the expense of loosing spatial lighting details lighting details

More photons can be shot at the More photons can be shot at the expense of performance lossexpense of performance loss

+ Significantly reduced the number of Significantly reduced the number of photons to be shot per framephotons to be shot per frame

+ Drastically reduced temporal aliasingDrastically reduced temporal aliasing

– Limited spatial resolution of mesh Limited spatial resolution of mesh reconstructed lightingreconstructed lighting

– For quickly changing indirect lighting For quickly changing indirect lighting temporal processing can be limitedtemporal processing can be limited

Spatial filtering can be performed at Spatial filtering can be performed at the expense of loosing spatial the expense of loosing spatial lighting details lighting details

More photons can be shot at the More photons can be shot at the expense of performance lossexpense of performance loss

Temporal processing: OnTemporal processing: On

Temporal processing: OffTemporal processing: Off

10,000 photons10,000 photons

25,000 photons25,000 photons

INFORMATIKINFORMATIKVisual AttentionVisual Attention

• Shrink the amount of visual information reaching the Shrink the amount of visual information reaching the eye to a manageable sizeeye to a manageable size

• Useful metaphor: Useful metaphor:

spotlight that enhances selected regionsspotlight that enhances selected regions

• Two components of visual attention:Two components of visual attention:

– bottom-up componentbottom-up component: fast; preattentive; primitive : fast; preattentive; primitive mechanism responding to color contrast, intensity mechanism responding to color contrast, intensity contrast, orientation, ...contrast, orientation, ...

– top-down componenttop-down component: slower; under cognitive : slower; under cognitive control; task-drivencontrol; task-driven

• Itti visual attention model – a popular choiceItti visual attention model – a popular choice

• Shrink the amount of visual information reaching the Shrink the amount of visual information reaching the eye to a manageable sizeeye to a manageable size

• Useful metaphor: Useful metaphor:

spotlight that enhances selected regionsspotlight that enhances selected regions

• Two components of visual attention:Two components of visual attention:

– bottom-up componentbottom-up component: fast; preattentive; primitive : fast; preattentive; primitive mechanism responding to color contrast, intensity mechanism responding to color contrast, intensity contrast, orientation, ...contrast, orientation, ...

– top-down componenttop-down component: slower; under cognitive : slower; under cognitive control; task-drivencontrol; task-driven

• Itti visual attention model – a popular choiceItti visual attention model – a popular choice


Visual AttentionVisual AttentionOff-line Scenario: [Yee’01]Off-line Scenario: [Yee’01]

Shading artifacts in “unattended” Shading artifacts in “unattended” image regions are likely to remain image regions are likely to remain unnoticed.unnoticed.• Use the visual attention model to decide Use the visual attention model to decide

the local quality of indirect lighting the local quality of indirect lighting computation in RADIANCEcomputation in RADIANCE

– Consider bottom-up component onlyConsider bottom-up component only Saliency Map [Itti’98]Saliency Map [Itti’98]

– Consider early vision path modeling Consider early vision path modeling Error Tolerance MapError Tolerance Map

• Speedup of irradiance caching: 3-9 timesSpeedup of irradiance caching: 3-9 times

• Further speedup by reusing the indirect Further speedup by reusing the indirect lighting for up to 10 in-between frameslighting for up to 10 in-between frames

Shading artifacts in “unattended” Shading artifacts in “unattended” image regions are likely to remain image regions are likely to remain unnoticed.unnoticed.• Use the visual attention model to decide Use the visual attention model to decide

the local quality of indirect lighting the local quality of indirect lighting computation in RADIANCEcomputation in RADIANCE

– Consider bottom-up component onlyConsider bottom-up component only Saliency Map [Itti’98]Saliency Map [Itti’98]

– Consider early vision path modeling Consider early vision path modeling Error Tolerance MapError Tolerance Map

• Speedup of irradiance caching: 3-9 timesSpeedup of irradiance caching: 3-9 times

• Further speedup by reusing the indirect Further speedup by reusing the indirect lighting for up to 10 in-between frameslighting for up to 10 in-between frames

Error Tolerance Map: Error Tolerance Map: higher tolerance higher tolerance

in brighter regions in brighter regions

Images: Yee et al.


Visual AttentionVisual AttentionInteractive Scenario: [Haber’01]Interactive Scenario: [Haber’01]

Shading artifacts of “unattended” glossy Shading artifacts of “unattended” glossy objects are likely to remain unnoticed.objects are likely to remain unnoticed.• Use visual attention models to schedule corrective Use visual attention models to schedule corrective

computations for glossy objects that are most likely to computations for glossy objects that are most likely to be “attended”:be “attended”:

– Consider both the saliency- and task-driven Consider both the saliency- and task-driven selection of those objectsselection of those objects

• Use progressive rendering approach: Use progressive rendering approach:

– Hierarchical sample splatting in the image spaceHierarchical sample splatting in the image space

– Cache samples and re-use them for similar viewsCache samples and re-use them for similar views

• Use multiple processors to increase the sample Use multiple processors to increase the sample numbernumber

Shading artifacts of “unattended” glossy Shading artifacts of “unattended” glossy objects are likely to remain unnoticed.objects are likely to remain unnoticed.• Use visual attention models to schedule corrective Use visual attention models to schedule corrective

computations for glossy objects that are most likely to computations for glossy objects that are most likely to be “attended”:be “attended”:

– Consider both the saliency- and task-driven Consider both the saliency- and task-driven selection of those objectsselection of those objects

• Use progressive rendering approach: Use progressive rendering approach:

– Hierarchical sample splatting in the image spaceHierarchical sample splatting in the image space

– Cache samples and re-use them for similar viewsCache samples and re-use them for similar views

• Use multiple processors to increase the sample Use multiple processors to increase the sample numbernumber


Visual Attention Visual Attention ProcessingProcessing

Open GL rendering Corrective splatting Converged solution: 5sOpen GL rendering Corrective splatting Converged solution: 5s

Saliency mapSaliency map


Warped old samplesWarped old samples

Adaptive SplattingAdaptive Splatting

Zoom Zoom inin

New samples:New samples: levels 1-2 levels 1-3 levels 1-2 levels 1-3 58 samples 188 samples58 samples 188 samples


Visual attention modelingVisual attention modeling+ Plausible way of scheduling the rendering Plausible way of scheduling the rendering

computationcomputation

– Can be expensive for interactive applicationsCan be expensive for interactive applications

– Possible problems with the robustness of saliency Possible problems with the robustness of saliency predictionprediction

Desirable mechanisms for overruling the model Desirable mechanisms for overruling the model predictionprediction

– May fail when the same visual material is seen May fail when the same visual material is seen repeatedlyrepeatedly

Visual attention modelingVisual attention modeling+ Plausible way of scheduling the rendering Plausible way of scheduling the rendering

computationcomputation

– Can be expensive for interactive applicationsCan be expensive for interactive applications

– Possible problems with the robustness of saliency Possible problems with the robustness of saliency predictionprediction

Desirable mechanisms for overruling the model Desirable mechanisms for overruling the model predictionprediction

– May fail when the same visual material is seen May fail when the same visual material is seen repeatedlyrepeatedly


Final Gathering for Two-Pass Final Gathering for Two-Pass Radiosity [Martin’99]Radiosity [Martin’99]• Exploiting temporal coherence in costly final gathering Exploiting temporal coherence in costly final gathering

• Texture moviesTexture movies used to represent space-time illumination used to represent space-time illumination

– Graphics hardware used to estimate the visibility at the Graphics hardware used to estimate the visibility at the level of texels in the texture movieslevel of texels in the texture movies

• Link classification in hierarchical line-space radiosityLink classification in hierarchical line-space radiosity

– Good links – error within tolerance limitGood links – error within tolerance limit Final gathering not required, mesh-based Final gathering not required, mesh-based

interpolationinterpolation

– Bad links – final gathering required Bad links – final gathering required Static linksStatic links – visibility update once per animation – visibility update once per animation

segmentsegment Dynamic linksDynamic links – visibility update for each frame – visibility update for each frame

• Exploiting temporal coherence in costly final gathering Exploiting temporal coherence in costly final gathering

• Texture moviesTexture movies used to represent space-time illumination used to represent space-time illumination

– Graphics hardware used to estimate the visibility at the Graphics hardware used to estimate the visibility at the level of texels in the texture movieslevel of texels in the texture movies

• Link classification in hierarchical line-space radiosityLink classification in hierarchical line-space radiosity

– Good links – error within tolerance limitGood links – error within tolerance limit Final gathering not required, mesh-based Final gathering not required, mesh-based

interpolationinterpolation

– Bad links – final gathering required Bad links – final gathering required Static linksStatic links – visibility update once per animation – visibility update once per animation

segmentsegment Dynamic linksDynamic links – visibility update for each frame – visibility update for each frame


+ Significant speedup of rendering: 2-7 Significant speedup of rendering: 2-7 timestimes

+ Reduced temporal aliasingReduced temporal aliasing

Popping may be visible when Popping may be visible when switching the precision of texture switching the precision of texture movies which is view-dependentmovies which is view-dependent

– Poor handling of non-Lambertian Poor handling of non-Lambertian surfacessurfaces

– High storage costsHigh storage costs

Also link history must be storedAlso link history must be stored

Clustering not supportedClustering not supported

+ Significant speedup of rendering: 2-7 Significant speedup of rendering: 2-7 timestimes

+ Reduced temporal aliasingReduced temporal aliasing

Popping may be visible when Popping may be visible when switching the precision of texture switching the precision of texture movies which is view-dependentmovies which is view-dependent

– Poor handling of non-Lambertian Poor handling of non-Lambertian surfacessurfaces

– High storage costsHigh storage costs

Also link history must be storedAlso link history must be stored

Clustering not supportedClustering not supportedImages: Martin et al.

INFORMATIKINFORMATIKSummarySummary

• Interactive global illumination Interactive global illumination

– One of the ultimate goals in the CG research for decades One of the ultimate goals in the CG research for decades

– Immense progress in recent yearsImmense progress in recent years

Faster/cheaper CPU/GPU Faster/cheaper CPU/GPU

Better algorithmsBetter algorithms

• Off-line global illumination for animationsOff-line global illumination for animations

– Reduction of the rendering cost per frame very importantReduction of the rendering cost per frame very important

– Can be achieved by better exploiting temporal coherenceCan be achieved by better exploiting temporal coherence

Better performanceBetter performance

Better quality - reduced temporal aliasingBetter quality - reduced temporal aliasing

• Successful solutions exist, but … still many things to doSuccessful solutions exist, but … still many things to do

• Interactive global illumination Interactive global illumination

– One of the ultimate goals in the CG research for decades One of the ultimate goals in the CG research for decades

– Immense progress in recent yearsImmense progress in recent years

Faster/cheaper CPU/GPU Faster/cheaper CPU/GPU

Better algorithmsBetter algorithms

• Off-line global illumination for animationsOff-line global illumination for animations

– Reduction of the rendering cost per frame very importantReduction of the rendering cost per frame very important

– Can be achieved by better exploiting temporal coherenceCan be achieved by better exploiting temporal coherence

Better performanceBetter performance

Better quality - reduced temporal aliasingBetter quality - reduced temporal aliasing

• Successful solutions exist, but … still many things to doSuccessful solutions exist, but … still many things to do

INFORMATIKINFORMATIKFuture Work DirectionsFuture Work Directions

Interactive applications:Interactive applications:• Framework unifying current methods designed to Framework unifying current methods designed to

solve very specific problemssolve very specific problems– Interactive ray tracing will benefit from increasing Interactive ray tracing will benefit from increasing

power of forthcoming CPUs, but …power of forthcoming CPUs, but …– the cost/performance of GPU increases with even the cost/performance of GPU increases with even

faster pace, and their programmability improves as faster pace, and their programmability improves as wellwell

High quality animations:High quality animations:• Affordable techniques supporting glossy effectsAffordable techniques supporting glossy effects• Design of a new renderer architecture emphasizing Design of a new renderer architecture emphasizing

on the temporal domainon the temporal domain

Interactive applications:Interactive applications:• Framework unifying current methods designed to Framework unifying current methods designed to

solve very specific problemssolve very specific problems– Interactive ray tracing will benefit from increasing Interactive ray tracing will benefit from increasing

power of forthcoming CPUs, but …power of forthcoming CPUs, but …– the cost/performance of GPU increases with even the cost/performance of GPU increases with even

faster pace, and their programmability improves as faster pace, and their programmability improves as wellwell

High quality animations:High quality animations:• Affordable techniques supporting glossy effectsAffordable techniques supporting glossy effects• Design of a new renderer architecture emphasizing Design of a new renderer architecture emphasizing

on the temporal domainon the temporal domain

State of the Art in Global Illumination for Interactive Applications and High-Quality Animations

Documents

Transcript of State of the Art in Global Illumination for Interactive Applications and High-Quality Animations