debevec
description
Transcript of debevec
www.debevec.orgwww.debevec.org
Paul Debevec, Tim Hawkins, Chris Tchou, H.P. Duiker, Westley Sarokin, and Mark Sagar
Paul Debevec, Tim Hawkins, Chris Tchou, H.P. Duiker, Westley Sarokin, and Mark Sagar
Acquiring the Reflectance Fieldof a Human Face
Acquiring the Reflectance Fieldof a Human Face
UC Berkeley / USC Institute for Creative Technologies / LifeF/XUC Berkeley / USC Institute for Creative Technologies / LifeF/X
Related WorkRelated WorkHanrahan and Krueger. Reflection from
Layered Surfaces due to Subsurface Scattering. SIGGRAPH 93
Bregler et al. Video Rewrite. SIGGRAPH 97
Guenter et al. Making Faces. SIGGRAPH 98
Pighin et al. Synthesizing Realistic Facial Expressions from Photographs. SIGGRAPH 98
Sagar et al. The Jester. SIGGRAPH 99 ET
Marschner et al. Reflectance Measurements of Human Skin. 1999
Hanrahan and Krueger. Reflection from Layered Surfaces due to Subsurface Scattering. SIGGRAPH 93
Bregler et al. Video Rewrite. SIGGRAPH 97
Guenter et al. Making Faces. SIGGRAPH 98
Pighin et al. Synthesizing Realistic Facial Expressions from Photographs. SIGGRAPH 98
Sagar et al. The Jester. SIGGRAPH 99 ET
Marschner et al. Reflectance Measurements of Human Skin. 1999
The Reflectance FieldThe Reflectance Field
The Reflectance FieldThe Reflectance Field
The Reflectance FieldThe Reflectance Field
The Reflectance FieldThe Reflectance Field
Ri( ui ,vi ,i ,i )Ri( ui ,vi ,i ,i )incident light fieldincident light field
The Reflectance FieldThe Reflectance Field
Rr ( ur ,vr ,r ,r )Rr ( ur ,vr ,r ,r )Ri( ui ,vi ,i ,i )Ri( ui ,vi ,i ,i )incident light fieldincident light field radiant light fieldradiant light field
The Reflectance FieldThe Reflectance Field
Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )8D reflectance field8D reflectance field
Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )Ri( ui ,vi ,i ,i ; ur ,vr ,r ,r )
4D Slices of the 8D
Reflectance Field
4D Slices of the 8D
Reflectance Field
The Light StageThe Light Stage
The Light Stage:60-second exposure
The Light Stage:60-second exposure
Light Stage DataLight Stage DataOriginal
Resolution: 6432
Original Resolution:
6432
Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97Lighting through image recombination: Haeberli ‘92, Nimeroff ‘94, Wong ‘97
Light Stage ResultsLight Stage Results
Environments from the Light Probe Image Gallerywww.debevec.org
Environments from the Light Probe Image Gallerywww.debevec.org
Reflectance FunctionsReflectance Functions
Lighting Reflectance FunctionsLighting Reflectance Functions
normalized light map
normalized light map
reflectance function
reflectance function
lighting productlighting product
rendered pixel
rendered pixel
1
1
DCT BasisDCT BasisSmith and Rowe. Compressed domain processing of JPEG-encoded images. 1996Smith and Rowe. Compressed domain processing of JPEG-encoded images. 1996
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Interactive Lighting DemoSIGGRAPH 2000 Creative Applications Laboratory
Changing the ViewpointChanging the Viewpoint
Reflection of Light from SkinReflection of Light from Skin
Specular Component:Color of light, shiny, brighter near grazing, maintains polarization
Subsurface Component:Color of skin, diffuse, desaturated near grazing, scrambles polarization
Specular Component:Color of light, shiny, brighter near grazing, maintains polarization
Subsurface Component:Color of skin, diffuse, desaturated near grazing, scrambles polarization
After Hanrahan ‘93After Hanrahan ‘93
Separating Reflectance Components using Crossed Polarizers
Separating Reflectance Components using Crossed Polarizers
Normal ImageNormal Image Subsurface ComponentSubsurface Component
Specular Component
Specular Component
Colorspace techniques - Sato ‘94, Nayar ‘97Colorspace techniques - Sato ‘94, Nayar ‘97
Transforming a Reflectance Function
Transforming a Reflectance Function
Subsurface ComponentSubsurface Component
Specular ComponentSpecular Component
Final RFFinal RF Comparison RFComparison RF
Shifted and ScaledSpecular
Shifted and ScaledSpecular
=> Torrance-Sparrow microfacet distribution
=> Torrance-Sparrow microfacet distribution
Surface Normal EstimateSurface Normal Estimate
Original RFOriginal RF
Point-Source ComparisonPoint-Source Comparison
Original ImageOriginal Image Novel ViewpointNovel Viewpoint
Spatially-VaryingReflectance Parameters
Spatially-VaryingReflectance Parameters
Surface Normals
n
Surface Normals
n
Diffuse Albedo
d
Diffuse Albedo
d
SpecularIntensity
ks
SpecularIntensity
ks
Specular Roughness
Specular Roughness
Compositing TestCompositing Test
Original ImageOriginal Image Light Probe Light Probe
Rendered FaceRendered Face Composite Composite
4. Ongoing Work4. Ongoing Work
• Animate the faces
• Capture more spectral samples
• Use high-speed cameras to achieve real-time capture
• Animate the faces
• Capture more spectral samples
• Use high-speed cameras to achieve real-time capture
5. Conclusion5. Conclusion
We have presented:
The light stage apparatus for capturing slices of the reflectance field of the human face
A technique for correctly relighting faces and objects with arbitrary illumination
A technique for extrapolating human reflectance to novel viewpoints
We have presented:
The light stage apparatus for capturing slices of the reflectance field of the human face
A technique for correctly relighting faces and objects with arbitrary illumination
A technique for extrapolating human reflectance to novel viewpoints
ThanksThanksDigital Media Innovation Program
Interactive Pictures Corporation
Alias|Wavefront
UCB Digital Digital Media/New Genre Program
ONR/BMDO
Cornell Program of Computer Graphics
Berkeley Millennium Project
Digital Media Innovation Program
Interactive Pictures Corporation
Alias|Wavefront
UCB Digital Digital Media/New Genre Program
ONR/BMDO
Cornell Program of Computer Graphics
Berkeley Millennium Project
and
Shawn Brixey, Bill Buxton, Larry Rowe, Jessica Vallot, Patrick Wilson, Melanie Levine, Eric Paulos, Christine Waggoner, Holly Cim, Eliza Ra, Bryan Musson,
David Altenau, Marc Levoy, Maryann Simmons, Henrik Wann Jensen, Don Greenberg, Pat Hanrahan, Randal Kleiser, Chris Bregler, Michael Naimark,
Dan Maas, Steve Marschner, and Kevin Binkert.
and
Shawn Brixey, Bill Buxton, Larry Rowe, Jessica Vallot, Patrick Wilson, Melanie Levine, Eric Paulos, Christine Waggoner, Holly Cim, Eliza Ra, Bryan Musson,
David Altenau, Marc Levoy, Maryann Simmons, Henrik Wann Jensen, Don Greenberg, Pat Hanrahan, Randal Kleiser, Chris Bregler, Michael Naimark,
Dan Maas, Steve Marschner, and Kevin Binkert.
In-plane Reflectometry Measurements
In-plane Reflectometry Measurements
Subsurface exhibits chromaticity falloff
Specular is monochromatic with Torrance-Sparrow microfacet behavior
Subsurface exhibits chromaticity falloff
Specular is monochromatic with Torrance-Sparrow microfacet behavior
SubsurfaceSubsurface
Subsurface + SpecularSubsurface + Specular
ii
rr
ii
rr
Reflectometry ExperimentReflectometry Experiment
The JesterSIGGRAPH 99 Electronic Theater
Mark Sagar et al. - LifeF/X, Inc.Performance and Text: Jessica Vallot
The JesterSIGGRAPH 99 Electronic Theater
Mark Sagar et al. - LifeF/X, Inc.Performance and Text: Jessica Vallot
Changing the ViewpointChanging the ViewpointModel from Structured Lighting
Traditional ApproachTraditional Approach
Derive reflectance parameters for each point on the face’s surface
Map the parameters onto a geometric model of the face
Render using traditional methods
Derive reflectance parameters for each point on the face’s surface
Map the parameters onto a geometric model of the face
Render using traditional methods
Reflectance Function Mosaic
Reflectance Function Mosaic