Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows
description
Transcript of Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows
0 00 00 0
0 0
Kalyan Sunkavalli, Todd Zickler, and Hanspeter Pfister Harvard UniversityVisibility Subspaces: Uncalibrated Photometric Stereo with Shadows
1 2 3 4 5NA
NB
NC
ND
L1 L2 L3 0 0L1 0 L3 L4 0L1 0 0 L4 L5
L1 L2 0 0 L5
1
2
3
4
5
Scene
B
A
C
D
Visibility regions Image matrix
=
Normals LightsImages
2
4
1
Shadows and Scene Appearance Lambertian Scene
ABCD
Dimensionality of scene appearance with (cast) shadowsThe set of all images of a Lambertian scene illuminated by any combination of n directional light sources lies in a linear space with dimension at most 3(2n).
Scene points with same visibility
Rank-3 subspaces of image matrix
(Visibility Subspaces)
Subspace normals
Estimated subspaces
True subspaces
Transformed normals
ResultsUncalibrated Photometric Stereo using Visibility Subspaces
Input images (4 of 12)
True subspaces
Estimated subspaces
Normals Depth
Input images (4 of 8)
True subspaces
Estimated subspaces
Normals Depth
Estimating Visibility SubspacesKey Idea: Find subspaces by constructing lighting bases from randomly sampled points, and checking how many other points project onto these bases with low error (RANSAC).
1. Sample three points in scene and estimate lighting basis:
2. Compute normals at all other points using this lighting basis:
3. Compute error of estimated lights and normals at every point:
4. Mark points with error as inliers. Compute lighting basis from all inliers.
5. Repeat 1 through 4. Mark largest inlier-set found as subspace with lighting .
6. Repeat 1 through 5 until all visibility subspaces have been recovered.
Images
Subspaces to surface normalsKey Idea: Estimate visibilities from subspaces, and use them to recover normals.• Subspace clustering recovers normals up to a linear 3X3 ambiguity in each subspace:
• From , compute visibility:• Using visibility, recover surface normals up to a single global linear 3X3 ambiguity by
solving an over-constrained system of linear equations for global lighting and subspace ambiguities :
For a Lambertian scene illuminated by lights , intensities at points in a subspace with visibility are given by:
3
5