Estimating 3D Attributes of Images from Shape Collections
1
Estimating 3D Attributes of Images from Shape Collections 1 Stanford Universiy 2 MPI Informatik 3 University College London Hao Su 1 , Qixing Huang 1 , Yangyan Li 1,2 , Niloy J. Mitra 3 , Leonidas J. Guibas 1 Preprocessing Joint Alignment Build initial correspondence final depth estimation iterative optimization init. point cloud Reconstruction camera estimation shape alignment deform shapes re -estimate depth matching in 3D Similar shape extraction init. correspondence Experimental Results Pipeline : utilizes network of similar (not identical) 3D objects to automatically reconstruct the depth information of a single image object. Input Image Similar Models Depth Recovery Kinect Scan principle space principle space eigen value eigen value Motivation: There are many more images (estimated in the trillions) than 3D models (estimated in the tens of millions). Estimating depth information on these images enables us to connect them with 3D shapes, allowing us to transfer complementary information back and forth. Key Idea: Shape deformation subspace can be learned from a network of shapes and then used to attribute a single image with depth. Shape Deformation Subspace: We learn a deformation model of each shape by its optimized deformations to other shapes. This deformation model serves as the regularizer for local shape space around each shape and is enforced in the reconstruction stage. Effect of Deformation Priors: The top/bottom rows show the results without/with the prior for a circular symmetric cup. Applications Depth based Relighting original original -30 -15 15 30 -30 -15 15 30 Novel View Synthesize
Transcript of Estimating 3D Attributes of Images from Shape Collections
Estimating 3D Attributes of Images from Shape Collections1Stanford Universiy 2MPI Informatik 3University College London
Hao Su1, Qixing Huang1, Yangyan Li1,2, Niloy J. Mitra3, Leonidas J. Guibas1
Preprocessing
Joint AlignmentBuild initial correspondence
final depthestimation
iterativeoptimization
fi l d thoptimization
init. point cloud
Reconstruction
camera estimationshape alignment
p g
deform shapes
re-estimate depth
matching in 3D
Similar shape extractioninit. correspondence
Experimental Results
Pipeline: utilizes network of similar (not identical) 3D objects to automatically reconstruct the depth information of a single image object.
Input Image Similar Models Depth Recovery Kinect Scan
principle space
principle space
eige
n va
lue
eige
n va
lue
Motivation: There are many more images (estimated in the trillions) than 3D models (estimated in the tens of millions). Estimating depth information on these images enables us to connect them with 3D shapes, allowing us to transfer complementary information back and forth.
Key Idea: Shape deformation subspace can be learned from a network of shapes and then used to attribute a single image with depth.
Shape Deformation Subspace: We learn a deformation model of each shape by its optimized deformations to other shapes. This deformation model serves as the regularizer for local shape space around each shape and is enforced in the reconstruction stage.its optimized deformations to other shapes. This deformation model serves as the regularizer for local shape space around each shape and is enforced in the reconstruction stage.shape space around each shape and is enforced in the reconstruction stage.shape space around each shape and is enforced in the reconstruction stage.
Effect of Deformation Priors: The top/bottom rows show the results without/with the prior for a circular symmetric cup.
Applications
Depth based Relighting
original
original-30 -15
15 30
-30 -15 15 30
Novel View Synthesize
