Torrance Sparrow Model of Reflectance + Oren Nayar Model of Reflectance
Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs
description
Transcript of Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs
![Page 1: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/1.jpg)
Inverse Global Illumination:Inverse Global Illumination:Recovering Reflectance Models of Real Recovering Reflectance Models of Real
Scenes from PhotographsScenes from Photographs
Inverse Global Illumination:Inverse Global Illumination:Recovering Reflectance Models of Real Recovering Reflectance Models of Real
Scenes from PhotographsScenes from Photographs
Computer Science Division
University of California at Berkeley
Computer Science Division
University of California at Berkeley
Yizhou Yu, Paul Debevec, Jitendra Malik & Tim Hawkins
![Page 2: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/2.jpg)
Image-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and Rendering
• 1st Generation---- vary viewpoint but not lighting– Recover geometry ( explicit or implicit )
– Acquire photographs
– Facade, Plenoptic Modeling, View Morphing, Lumigraph, Layered Depth Images, (Light Field Rendering) etc.
• 1st Generation---- vary viewpoint but not lighting– Recover geometry ( explicit or implicit )
– Acquire photographs
– Facade, Plenoptic Modeling, View Morphing, Lumigraph, Layered Depth Images, (Light Field Rendering) etc.
![Page 3: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/3.jpg)
Image-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and RenderingImage-based Modeling and Rendering
• Photographs are not Reflectance Maps !
• 2nd Generation---- vary viewpoint and lighting for non-diffuse scenes– Recover geometry
– Recover reflectance properties
– Render using light transport simulation
• Photographs are not Reflectance Maps !
• 2nd Generation---- vary viewpoint and lighting for non-diffuse scenes– Recover geometry
– Recover reflectance properties
– Render using light transport simulation
Illumination Radiance
Reflectance
![Page 4: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/4.jpg)
Previous WorkPrevious WorkPrevious WorkPrevious Work
• BRDF Measurement in the Laboratory– [ Ward 92 ], [Dana, Ginneken, Nayar & Koenderink 97]
• Isolated Objects under Direct Illumination– [ Sato, Wheeler & Ikeuchi 97 ]
• Isolated Objects under General Illumination– [ Yu & Malik 98], [ Debevec 98]
• BRDF Measurement in the Laboratory– [ Ward 92 ], [Dana, Ginneken, Nayar & Koenderink 97]
• Isolated Objects under Direct Illumination– [ Sato, Wheeler & Ikeuchi 97 ]
• Isolated Objects under General Illumination– [ Yu & Malik 98], [ Debevec 98]
![Page 5: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/5.jpg)
The ProblemThe ProblemThe ProblemThe Problem
• General case of multiple objects under mutual illumination has not been studied.
![Page 6: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/6.jpg)
Global IlluminationGlobal IlluminationGlobal IlluminationGlobal Illumination
Reflectance Properties
Radiance Images
Geometry Illumination
![Page 7: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/7.jpg)
Inverse Global IlluminationInverse Global IlluminationInverse Global IlluminationInverse Global Illumination
Reflectance Properties
Radiance Images
Geometry Illumination
![Page 8: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/8.jpg)
Input Radiance ImagesInput Radiance ImagesInput Radiance ImagesInput Radiance Images
[ Debevec & Malik 97]http://www.cs.berkeley.edu/~debevec/HDR
![Page 9: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/9.jpg)
In Detail ... In Detail ... In Detail ... In Detail ...
![Page 10: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/10.jpg)
Geometry and Camera PositionsGeometry and Camera PositionsGeometry and Camera PositionsGeometry and Camera Positions
![Page 11: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/11.jpg)
Light SourcesLight SourcesLight SourcesLight Sources
![Page 12: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/12.jpg)
Synthesized ImagesSynthesized ImagesSynthesized ImagesSynthesized Images
Original Lighting Novel Lighting
![Page 13: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/13.jpg)
OutlineOutlineOutlineOutline
• Diffuse surfaces under mutual illumination
• Non-diffuse surfaces under direct illumination
• Non-diffuse surfaces under mutual illumination
• Diffuse surfaces under mutual illumination
• Non-diffuse surfaces under direct illumination
• Non-diffuse surfaces under mutual illumination
![Page 14: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/14.jpg)
Lambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual IlluminationLambertian Surfaces under Lambertian Surfaces under Mutual IlluminationMutual Illumination
j
ijjiii FBEB j
ijjiii FBEB
• Bi, Bj, Ei measured
• Form-factor Fij known
• Solve for diffuse albedo
• Bi, Bj, Ei measured
• Form-factor Fij known
• Solve for diffuse albedo i
iB
jBijF
Source
Target
![Page 15: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/15.jpg)
Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]Parametric BRDF Model [ Ward 92 ]
Isotropic Kernel
Anisotropic Kernel
NHi
r
),(
Ksd
2
22
4
]/tan[exp
coscos
1),(
ri
K
yx
yx
ri
K
4
)]/sin/cos(tan[exp
coscos
1),(
22222
( 3 parameters)
( 5 parameters)
![Page 16: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/16.jpg)
Non-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect IlluminationNon-diffuse Surfaces underNon-diffuse Surfaces underDirect IlluminationDirect Illumination
2
,,)),(( min arg iisi
i
di IKIL
sd
2
,,)),(( min arg iisi
i
di IKIL
sd
NH
iisid
i IKIL )),((
iisid
i IKIL )),((
P1P2
P1
P2
![Page 17: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/17.jpg)
Non-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual IlluminationNon-diffuse Surfaces under Non-diffuse Surfaces under Mutual IlluminationMutual Illumination
• LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )
• LPiAj is not known. ( unlike diffuse case, where LPiAj = LCkAj )
j j
APCAPsAPAPdPC jivjijijiivKLFLL
j jAPCAPsAPAPdPC jivjijijiiv
KLFLL Cv
Ck
Aj
Pi
LPiAj
LCkAj
LCvPi
Source
Target
![Page 18: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/18.jpg)
Solution: iteratively estimate Solution: iteratively estimate specular component.specular component.Solution: iteratively estimate Solution: iteratively estimate specular component.specular component.
jikjkji APCACAP SLL jikjkji APCACAP SLL
• Initialize
• Repeat– Estimate BRDF parameters for each surface
– Update and
• Initialize
• Repeat– Estimate BRDF parameters for each surface
– Update and
0jik APCS 0jik APCS
jik APCS jik APCS jiAPL
![Page 19: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/19.jpg)
Estimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference SEstimation of Specular Difference S
• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.
• Similarly for
• Difference gives S
• Estimate specular component of by Monte Carlo ray-tracing using current guess of reflectance parameters.
• Similarly for
• Difference gives S Cv
Ck
Aj
Pi
LPiAj
LCkAj
LCvPi
LPiAj
LCkAj
![Page 20: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/20.jpg)
Recovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo MapsRecovering Diffuse Albedo Maps
• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.
•
•
• Specular properties assumed uniform across each surface, but diffuse albedo allowed to vary.
•
•
)()()( xLxLxL sd
)(/)()( xIxLx dd
![Page 21: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/21.jpg)
ResultsResultsResultsResults
• A simulated cubical room• A simulated cubical room
![Page 22: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/22.jpg)
Results for the Simulated CaseResults for the Simulated CaseResults for the Simulated CaseResults for the Simulated Case
Diffuse Albedo
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 60
0.05
0.1
0.15
0.2
0.25
0.3
0.35
1 2 3 4 5 6
Specular Roughness
![Page 23: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/23.jpg)
ResultsResultsResultsResults
• A real conference room• A real conference room
![Page 24: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/24.jpg)
Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting Real vs. Synthetic for Original Lighting
Real
Synthetic
![Page 25: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/25.jpg)
Diffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different PositionsDiffuse Albedo Maps of Identical Diffuse Albedo Maps of Identical Posters in Different PositionsPosters in Different Positions
Poster A Poster B Poster C
![Page 26: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/26.jpg)
Inverting Color BleedInverting Color BleedInverting Color BleedInverting Color Bleed
Input Photograph Output Albedo Map
![Page 27: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/27.jpg)
Real vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel LightingReal vs. Synthetic for Novel Lighting
Real
Synthetic
![Page 28: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/28.jpg)
VideoVideoVideoVideo
![Page 29: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/29.jpg)
AcknowledgmentsAcknowledgmentsAcknowledgmentsAcknowledgments
• Thanks to David Culler and the Berkeley NOW project, Tal Garfinkel, Gregory Ward Larson, Carlo Sequin.
• Supported by ONR BMDO, the California MICRO program, Philips Corporation, Interval Research Corporation and Microsoft Graduate Fellowship.
• Thanks to David Culler and the Berkeley NOW project, Tal Garfinkel, Gregory Ward Larson, Carlo Sequin.
• Supported by ONR BMDO, the California MICRO program, Philips Corporation, Interval Research Corporation and Microsoft Graduate Fellowship.
![Page 30: Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs](https://reader036.fdocuments.in/reader036/viewer/2022062409/5681514c550346895dbf6f04/html5/thumbnails/30.jpg)
ConclusionsConclusionsConclusionsConclusions
• A digital camera can undertake all the data acquisition tasks involved.
• Both specular and high resolution diffuse reflectance properties can be recovered from photographs.
• Reflectance recovery can re-render non-diffuse real scenes under novel illumination as well as from novel viewpoints.
• A digital camera can undertake all the data acquisition tasks involved.
• Both specular and high resolution diffuse reflectance properties can be recovered from photographs.
• Reflectance recovery can re-render non-diffuse real scenes under novel illumination as well as from novel viewpoints.