Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.
-
Upload
primrose-payne -
Category
Documents
-
view
222 -
download
0
Transcript of Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.
![Page 1: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/1.jpg)
Statistical and numerical techniquesfor photorealistic image synthesis
Kartic Subr
![Page 2: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/2.jpg)
Who am I?
• Born in India– Bangalore University (Bachelor of Engineering) 2001– Hewlett Packard, India/Singapore
• 6 years in USA– PhD, University of California Irvine, 2008– Advisor: Jim Arvo (PhD Yale University), pioneered methods in light transport
• 2 years in France– Post doctoral researcher, ARTIS, INRIA-Grenoble (2008-2010)
![Page 3: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/3.jpg)
My goal: Generating realistic visuals
Gustave Courbet, Stone-Breakers, 1849.
Wilhelm Oswald Gustav Achenbach, Abendstimmung in der Campagna, 1850.
Realism in art
![Page 4: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/4.jpg)
Gustave Courbet, Stone-Breakers, 1849.
Wilhelm Oswald Gustav Achenbach, Abendstimmung in der Campagna, 1850.
Realism in art
Photograph: Nicéphore Niépce, 1826
My goal: Generating realistic visuals
![Page 5: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/5.jpg)
Notion of “realism” depends on technology
Gerhard Richter, 1983
Pedro Campos
Hyperrealism
![Page 6: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/6.jpg)
Realistic image synthesis
![Page 7: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/7.jpg)
Image synthesis involves light transport
Image ?
Digital models of scene (geometry + materials)
Light sources
Virtual camera
![Page 8: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/8.jpg)
Image synthesis adds visual impact
AvatarCaptured video+ digital model
Digital model
![Page 9: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/9.jpg)
Applications of image synthesis
Defense
Advertising
Entertainment
Virtual prototyping
Biomedical imaging
![Page 10: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/10.jpg)
Multidisciplinary nature of the problem
• Physically based optical simulations
• Mathematical tools for analysis
• Numerical techniques for light transport solution
• Understanding biological processeseg. Perception et cognition
![Page 11: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/11.jpg)
Reflection of light is an integration
![Page 12: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/12.jpg)
Light transport: multi-domain integration
• Combinatorial explosion from sampling each domain
Image spaceAperture
Exposure time
Visible spectrum
Reflectance Direct illumination Indirect illumination
[Efficient sampling strategies for Monte Carlo integration (my PhD thesis)]
![Page 13: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/13.jpg)
Talk outline
• Recent contributions– Simulating defocus– Rendering translucent materials
• Research plan– Core problems in image synthesis– Model representation and abstraction
![Page 14: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/14.jpg)
My contributions: Fourier depth of field
![Page 15: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/15.jpg)
Defocus blur is important in photography
![Page 16: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/16.jpg)
Defocus is due to aperture integraion
Image Aperture
Pixel p
Lens
![Page 17: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/17.jpg)
Defocus
Image Aperture
Pixel p
Lens Scene
Pixel p
![Page 18: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/18.jpg)
Monte Carlo estimation of aperture integral
Image Aperture
NA primary rays per pixel
Integrateat p
![Page 19: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/19.jpg)
Aperture integration is very costly
ImageAperture
NP pixels
NP x NA Primary rays
NA Aperture samples
![Page 20: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/20.jpg)
64 x #primary rays of the pinhole image
Paradox: Blurry image is costlier to compute!
![Page 21: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/21.jpg)
Observation 1: Image
Blurry regionsshould not require
dense samplingof the image
![Page 22: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/22.jpg)
Observation 2: Lens
Regions in focusshould not requireprofuse samplingof the lens for diffuse objects
![Page 23: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/23.jpg)
Fourier depth of field
• Fourier domain analysis of finite aperture cameras• Adaptive sampling• Speedup of around 20 over existing algorithms
[ACM Transactions on Graphics 2009. Presented at ACM SIGGRAPH 09]
Collaborators: MIT
![Page 24: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/24.jpg)
My contributions: Translucent materials
![Page 25: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/25.jpg)
TranslucentOpaque
Translucency: Sub-surface scattering
• Brute force Monte Carlo: prohibitively expensive• Diffusion approximations: severe constraints on geometry
![Page 26: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/26.jpg)
Finite difference method on new domain
• Approximation: diffusion equation
• Domain: Dual graph of tetrahedralization
Diffuse flux
![Page 27: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/27.jpg)
Rendering translucent materials
[Computer Graphics Forum 2010. To be presented at Eurographics 2010]
[Collaborators: Microsoft Research, Tsinghua University]
• Arbitrary geometry• Heterogenous materials• Dynamically deforming shapes• In real-time!
![Page 28: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/28.jpg)
Research program
Realistic image synthesisModel representation
and abstraction
![Page 29: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/29.jpg)
1. Realistic image synthesis
• Bandwidth driven sampling– Transport of local light field spectrum– Derive spatial / angular sampling rates– co-advising PhD student Laurent Belcour (ARTIS)
• Importance vs radiance– Tracing from eye vs tracing from light– Monte Carlo matrix chain multiplication
Short-term
Long-term
![Page 30: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/30.jpg)
Importance vs radiance
Radiance
![Page 31: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/31.jpg)
Importance vs radiance
Importance
![Page 32: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/32.jpg)
Importance vs radiance
![Page 33: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/33.jpg)
MC matrix-chain product estimator
![Page 34: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/34.jpg)
MC matrix-chain product estimator
Related to optimal matrix chain multiplication using dynamic programming?
![Page 35: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/35.jpg)
2. Model representation and abstraction
• Abstracting detail in geometry– First step: images (published at SIGGRAPH Asia 09)
• Alternate representation– Voxel data to represent geometry and materials
Short-term
Long-term
![Page 36: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/36.jpg)
Detail = oscillations between extrema
Input
Local maxima
Local minima
![Page 37: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/37.jpg)
Image multiscale decomposition
+
+
Medium
Pixels
Intensity
Input
Fine
Coarse
1D
![Page 38: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/38.jpg)
Allows smoothing high-contrast detail
Input
Smoothed
![Page 39: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/39.jpg)
Thank you!• Collaborators
– Established • MIT, USA• Microsoft Research• Tsinghua University, China• University of California, Irvine
– Current• Cornell University, USA• University of California, Berkeley
– Potential• Indian Institute of Information Technology
• International journal publications– Computer Graphics Forum 2010: Translucent materials. 4th author of 6– TOG 2009: Multiscale image decomposition. 1st author of 3– TOG 2009: Fourier Depth of Field. 2nd author out of 5
• Refereed international conference papers– Pacific Graphics 2007: Statistical hypotheses. 1st author of 2– Interactive raytracing 2007: Steerable importance sampling. 1st author of 2– ICIAP 2005: Contrast enhancement. 1st author of 3
![Page 40: Statistical and numerical techniques for photorealistic image synthesis Kartic Subr.](https://reader036.fdocuments.in/reader036/viewer/2022062314/56649ee05503460f94bf0976/html5/thumbnails/40.jpg)
Merci !• Collaborators
– Established • MIT, USA• Microsoft Research• Tsinghua University, China• University of California, Irvine• LJK Grenoble
– Current• Cornell University, USA• University of California, Berkeley
– Potential• Indian Institute of Information Technology
• International journal publications– Computer Graphics Forum 2010: Translucent materials. 4th author of 6– TOG 2009: Multiscale image decomposition. 1st author of 3– TOG 2009: Fourier Depth of Field. 2nd author out of 5
• Refereed international conference papers– Pacific Graphics 2007: Statistical hypotheses. 1st author of 2– Interactive raytracing 2007: Steerable importance sampling. 1st author of 2– ICIAP 2005: Contrast enhancement. 1st author of 3
• Teaching– Columbia University, USA (120 h)– University of California, Irvine (360 h)
• Industry– Rhythm and Hues Studios– NVIDIA Corporation– Hewlett Packard