Post on 05-Apr-2018
7/31/2019 PhD Des Pin a Michael Online Presentation
1/50
7/31/2019 PhD Des Pin a Michael Online Presentation
2/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 2
Contents
Introduction
Illumination
Problem statement
Approach
Proposed algorithm Results
Tonemapping
Problem statement
Approach
Proposed algorithm
Results
Conclusions & Future Work
7/31/2019 PhD Des Pin a Michael Online Presentation
3/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 3
introduction
7/31/2019 PhD Des Pin a Michael Online Presentation
4/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 4
Computer graphics (CG)
What is CG?It is the field where one utilizes digital data to generateimages
Applications Training - Simulators
military medical aviation
Entertainment movies
computer games Virtual Worlds
virtual museums virtual cities
Etc
7/31/2019 PhD Des Pin a Michael Online Presentation
5/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 5
3D Computer Graphics
Geometric modeling
the representation of the shapes of objects in the scene
Animation
the motion and deformation of the objects in the scene
Rendering
the process of producing 2D images from the description of the 3Dscene
Computer graphics (CG)
Rendering 2D image
Rendering+
7/31/2019 PhD Des Pin a Michael Online Presentation
6/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 6
Object definition
Local spaceModeling
transformationsWorld space
Viewingtransformations
View space
Projectionontoa
unitcube&Clipping
Clip spaceScreenmapping
Projected spaceRasterization& Viewport mapping
Display space
Rendering Pipeline
Compose scene,Define view
& Lighting conditions
Pixel Shading
Vertex Shading
7/31/2019 PhD Des Pin a Michael Online Presentation
7/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 7
Importance of illumination
Illumination in real world:
Light stimulates sensors to see
Without light human eye can not see
Illumination in Computer Graphics:
Provides spatial and geometric information Increases realism in the images
no illumination spatial and geometric information simulated
measured
7/31/2019 PhD Des Pin a Michael Online Presentation
8/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 8
1 2 3
4 5 6
7 8 9
1 2 3
4 5 6
7 8 9
7/31/2019 PhD Des Pin a Michael Online Presentation
9/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 9
Motivation
Contribution
Improve rendering for real-time applications
Illumination
Tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
10/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 10illumination
7/31/2019 PhD Des Pin a Michael Online Presentation
11/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 11
illumination at each point of an object depends on: Incident light energy
Reflectance properties of the object material
BRDF: Bidirectional Reflectance Distribution Functiondefines how light coming from a given incoming
direction is reflected towards an outgoing direction
Parameters affecting illumination
7/31/2019 PhD Des Pin a Michael Online Presentation
12/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 12
Rendering equation
p
Outgoing Light
BRDF x Incident Light
+ Reflected Light= Emitted Light
Radiance
7/31/2019 PhD Des Pin a Michael Online Presentation
13/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 13
Computational expensive process
Emphasis on realism - global illumination (offline methods)
Emphasis on frame rate - local illumination
Solving the rendering equation
Raytracing Radiosity Photon mapping
Shadow maps Shadow volumes
7/31/2019 PhD Des Pin a Michael Online Presentation
14/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 14
IlluminationProblem statement
Towards real-time realism Computation of direct lighting only
Taking into account incident radiance from allover the directions
represented using environment maps distant lighting
7/31/2019 PhD Des Pin a Michael Online Presentation
15/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 15
IlluminationProblem statement
Huge number of incident light directions (e.g 10
3
) Run-time computation of not real-time performance
Precomputations with what cost?
PRT [Sloan]at each vertex
static scenes + memory
Precomputedshadow fields [Zhou]rigid objects + memory
Real-timesoft shadows [Ren]
geometry approx.
7/31/2019 PhD Des Pin a Michael Online Presentation
16/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 16
Proposed illumination algorithmApproach
Precompute the in a way that isindependent of the geometry of the scene
p depends on
Irradiance
7/31/2019 PhD Des Pin a Michael Online Presentation
17/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 17
Proposed illumination algorithmApproach
FullsphereIrradiance
Irradiance
7/31/2019 PhD Des Pin a Michael Online Presentation
18/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 18
Proposed illumination algorithmApproach
FI Factorization
Fullsphere Irradiance Vector FIV(x,y,z)
FIV
x
FIVy
compute FIV
p
p
.
p
Only a dot product !!(fast computation)
Valid for any receiver(only a 3D vector save memory)
7/31/2019 PhD Des Pin a Michael Online Presentation
19/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 19
Compute and not
Use FIV as an intermediate step
Break down the problem
Precomputations for dynamic scenes
p
Proposed illumination algorithmOverview
p
)( pp NI
= -p p
)(, pptot NI
)(, ppocc NI
p
pI
pFI
p .
p
1 2
d ll l h
7/31/2019 PhD Des Pin a Michael Online Presentation
20/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 20
.p
IrradianceMap
)(, pptot
NI
.
Proposed illumination algorithmPreprocessing
1
p
? )(, pptot NI
Precompute
for all
p
2
for eachsample position
(i, j, k)
EnvOcc
for eachEnvOcc
(i, j, k)
compute FIV
FIV (x, y, z)
FIVs texture
(~ 1MB)for 64x33x15 samples
(72KB)
P d ill i i l i h
7/31/2019 PhD Des Pin a Michael Online Presentation
21/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 21
For each pixel p of a receiver in the image:
1. Get total irradiance assuming no occlusions
2. Compute occluded irradiance
3. Compute unoccluded irradiance
4. Shade the pixel
Proposed illumination algorithmRuntime: shading
)( pp NI = -p p
)(, pptot NI
)(, ppocc NI
1 2
. .p.
)(, pptot
NI
Irradiance Map
)(, ppocc
NI
FIV of. )(, ppocc NFI
)()()( ,, ppoccpptotpp NININI
FIVs texture
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
22/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 22
Proposed illumination algorithmRuntime: special cases
Partialoccluders
Approximate FIV
Overlappedoccluders
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
23/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 23
Proposed illumination algorithmRuntime: self-shadows
p
p
shadows self-shadows
pp
p
1 2
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
24/5015/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 24
Proposed illumination algorithmResults
GPU implementation of the algorithm (GLSL)
Per-pixel illumination Speed-up the computations
Intel Core 2 Duo 2.4GHz 2GB RAM NVIDIA GeForce 8800 GTS (512MB Ram)
Environment map resolution: 32x32x6 pixels Higher resolution no difference at run-time fps
only the preprocessing time
Window resolution: 512x512
Trilinear interpolation of the 8 nearest FIVs
Comparing with ground truth solution Brute force solution Normalized Mean Square Error (NRMSE)
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
25/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 25
Proposed illumination algorithmResults
Proposed algorithm result (without rendering the occluder)
Result of brute force/ground truth solution
Difference between the two
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
26/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 26
Proposed illumination algorithmResults
Different Number of Samples
Different Environment Maps(samples #: 256x129x30)
(, , ) = (32,17,15)NRMSE = 0.023
0.28 MBytes
(, , ) = (64,33,30)NRMSE = 0.008
2.17 MBytes
(, , ) = (256,129,30)NRMSE = 0.007
34 MBytes
1 point lightNRMSE = 0.1
2 area lightsNRMSE = 0.035
Eucalyptus GroveNRMSE = 0.022
P d ill i ti l ith
7/31/2019 PhD Des Pin a Michael Online Presentation
27/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 27
Proposed illumination algorithmResults
Multiple occluders
Partial Occluders
NRMSE = 0.019 NRMSE = 0.019
NRMSE = 0.026
Correction: OFF
NRMSE = 0.019
Correction: ON
P oposed ill mination algo ithm
7/31/2019 PhD Des Pin a Michael Online Presentation
28/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 28
Proposed illumination algorithmResults
Other cases
Deformable/Dynamic receiver
Complex Scene10 objects
~ 300K vertices
Self-shadows
only direct direct &self-shadows
difference
Proposed illumination algorithm
7/31/2019 PhD Des Pin a Michael Online Presentation
29/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 29
Precomputation time & memory requirements
Memory requirements: independent of the number of vertices
linear to the number of samples
Precomputations time:(implemented on CPU)
linear to the number of samples
Proposed illumination algorithmResults
(, , ) = (32,17,15)
NRMSE = 0.0233 mins / 0.28MB
Proposed illumination algorithm
7/31/2019 PhD Des Pin a Michael Online Presentation
30/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 30
Combination of Objects
Number of FPS: Independent of the number of vertices of the object
Depends on the number of objects in the scene
Memory requirements: Depends on the number of different types of objects
multiple times the same object does not increase requirements
Proposed illumination algorithmResults
PRT Precomputed Dynamic FIV
7/31/2019 PhD Des Pin a Michael Online Presentation
31/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 31
COMPARISONWITH OTHER
METHODS
PRT
Sloan
SIGGRAPH
PrecomputedShadow Fields
Zhou
SIGGRAPH
DynamicSoft
Shadows
RenACM TOC
FIV
this thesis
Env. Map (EM)Frequencies
Low Low (dynamic scenes)High (static scenes)
Low All
Dynamic scenes Rigid Rigid DeformableDeformable
receivers & rigidoccluders
Geometry
approx.
No No Yes No
Memoryrequirements
dif. obj/vert./EM
>GB
50MB
7 / 30K / low
500 MB
2 / 70K / high
They do notdemonstrateany results
7/31/2019 PhD Des Pin a Michael Online Presentation
32/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 32
Proposed illumination algorithmHDR illumination
High-quality illumination
incident radiance natural values can be used
Real-world
Vast range of luminance values
Examples: sunlight: 105 cd/m2
indoor lighting: 102 cd/m2
starlight: 10-3 cd/m2
~10 orders of magnitude in luminance values
Within a scene we may have High Dynamic Range (HDR) ofluminance values
dynamic range: ratio of the highest scene luminance to the lowestscene luminance
Natural illumination
Take into account HDR incident radiance from the environment
Proposed Illumination Algorithm
7/31/2019 PhD Des Pin a Michael Online Presentation
33/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 33
HDR images can not be displayed on standard monitors (LDR)
A final pass is needed to map HDR values to LDR
Tonemapping
Proposed Illumination AlgorithmHDR illumination
Illuminationalgorithm
HDRilluminated
sceneHDR incidentradiance
3D scene
Angeloktistichurch in KitiLarnaca, Cyprus
Tonemapping
http://material/kiti_rect.hdrhttp://material/kiti_rect.hdrhttp://material/kiti_rect.hdrhttp://material/kiti_rect.hdrhttp://material/kiti_rect.hdrhttp://material/kiti_rect.hdr7/31/2019 PhD Des Pin a Michael Online Presentation
34/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 34tonemapping
Tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
35/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 35
TonemappingProblem statement
Tonemapping operators
Global operators Local operators
Global operators Apply same operation to all pixels Fast Computation Moderate quality results
Local operators Take into consideration the local properties
of individual pixels Preserve the local contrast reproduction
local and global component
Better quality results More computationally expensive Much slower than global operators
We need High frame rates High-quality
Proposed selective tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
36/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 36
Proposed selective tonemappingApproach
Apply the computational expensive local component of TMO
in a selective way Only on the high contrast areas (important areas)
edge detection algorithm
Reduce expensive computations
Maintain the quality
Proposed selective tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
37/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 37
globalcomponent
LUM2RGB
RGB2LUM
HDR input (RGB) Luminance
high contrastidentification
high contrast
apply localcomponent
keep the originalLUM values
LDR input (RGB)
Proposed selective tonemappingFramework
Proposed selective tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
38/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 38
Proposed selective tonemappingResults
HDR image
Selective TM Local TM
Important areas &superimposed VDP map
VDP: Visual Difference Predictor
GTM
STM
LTM
Proposed selective tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
39/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 39
Proposed selective tonemappingResults
Sel
ectiveTonem
apping
LocalTonemapp
ing
SpheronVR SpheronVR
Proposed selective tonemapping
7/31/2019 PhD Des Pin a Michael Online Presentation
40/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 40
Proposed selective tonemappingResults
Real-time fps for high resolution images Much faster than LTM (~3 times)
Better quality results than GTM
Better than GTM taking into account quality & time
7/31/2019 PhD Des Pin a Michael Online Presentation
41/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 41
Unified Rendering Pipeline
Illuminationalgorithm
HDR
illuminatedscene
HDR incidentradiance
3D scene
Tonemapping
LDR
illuminatedscene
Unified Rendering Pipeline
7/31/2019 PhD Des Pin a Michael Online Presentation
42/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 42
Unified Rendering PipelineResults
Visual error of illumination algorithm decreases or increases aftertonemapping? (error before TM VS error after TM)
Different number of sampleswithout
TMVDPmap
(without)
with
TM
VDPmap
(with)
Ground
truth
32x17x15
64x33x30
256x129x30
Unified Rendering Pipeline
7/31/2019 PhD Des Pin a Michael Online Presentation
43/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 43
Unified Rendering PipelineResults
Different environment maps
with-out
TM
Groundtruth
FIV
VDP map
withTM
Groundtruth
FIV
VDP map
7/31/2019 PhD Des Pin a Michael Online Presentation
44/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 44
conclusions
& future work
7/31/2019 PhD Des Pin a Michael Online Presentation
45/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 45
Conclusions
Illumination Fullsphere Irradiance (new notion)
Fullsphere Irradiance Vector (FIV)
Integrate incident light energy within a 3D
vector independent of
arbitrary number of light sources
Use FIVs to illuminate the scene Soft-shadows
Real-time
High-quality results
Moderate memory requirements
7/31/2019 PhD Des Pin a Michael Online Presentation
46/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 46
Conclusions
Tonemapping
Selective Tonemapping Framework
Speed up local TMOs
Real-time for high resolution frames
Keep quality (perceptually) of original local TMO
Modular framework
use any other local TMO
use other algorithm for important areas identificationstep
Makes illumination algorithm more tolerant toerrors
7/31/2019 PhD Des Pin a Michael Online Presentation
47/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 47
Future work
Illumination Specular reflections
no direct way to do that
Adaptive sampling
Online computations of FIVs GPU implementation
only at samples needed
First bounce of inter-reflections
Tonemapping
Exploit more temporal coherence between frames reuse parts of high contrast areas map
High contrast areas map Important areas map
what is important? e.g. the main character in a game
Interactive/real-time rates
7/31/2019 PhD Des Pin a Michael Online Presentation
48/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 48
Publications
1. Despina Michael and Yiorgos Chrysanthou. Automatic high level avatar guidance basedon affordance of movement, Eurographics 2003, In Proceedings Interactive Demos andPosters, pp 221-226, Published by the Eurographics Association, 2003.
2. Benjamin Roch, Alessandro Artusi, Despina Michael, Yiorgos Chrysanthou, andAlan Chalmers. Interactive local tone mapping operator with the support ofgraphics hardware In ACM Proceedings, SCCG Conference 2007, Published byACM, 2007.
3. Petros Patias, Yiorgos Chrysanthou, Stella Sylaiou, Charis Georgiadis, Despina Michael,and Stratos Stylianidis. The development of an e-museum for contemporary arts. InVirtual Systems and Multimedia, VSMM08, 2008.
4. Samuel Obadan, Andreas Gregoriades, Harris Michail, Vicky Papadopoulou, DespinaMichael. A Robotic System for Home Security Enhancement, ICOST 2010.
5. Despina Michael and Yiorgos Chrysanthou, Fullsphere -Irradiance factorizationfor real- time all-frequency illumination for dynamic scenes. ComputerGraphics Forum Journal, accepted for publication, 2010.
6. Despina Michael, Panagiotis Zaharias and Yiorgos Chrysanthou. A virtual tour of theWalls of Nicosia: An assessment of childrens' experience and learning effectiveness,
VAST 2010.
7. Despina Michael, Nektarios Pelekanos, Isabelle Chrysanthou, Panagiotis Zaharias andYiorgos Chrysanthou, Comparative Study of Interactive Systems in a Museum,submitted EuroMed 2010.
8. Selective Tonemapping, to be submitted in a CG journal.
7/31/2019 PhD Des Pin a Michael Online Presentation
49/50
15/07/2010 Real-time High Quality HDR Illumination and Tonemapped Rendering 49
Acknowledgments
Yiorgos Chrysanthou
Alessandro Artusi, CaSToRC The Cyprus Institute, Cyprus
Benjamin Roch, Technical University of Vienna, Austria
CG Group @ CS.UCY
CS.UCY
Family
Aimilia, Andreas, Anna, Christos, Costas, Georgios I., George S.,Marios, Nearchos, Pyrros, Ritsa, Vicky (& little baby coming soon)
PhD examination committee
7/31/2019 PhD Des Pin a Michael Online Presentation
50/50
Thank you for your attention!