Estimating 3D Attributes of Images from Shape Collections1Stanford Universiy 2MPI Informatik 3University College London

Hao Su1, Qixing Huang1, Yangyan Li1,2, Niloy J. Mitra3, Leonidas J. Guibas1

Preprocessing

Joint AlignmentBuild initial correspondence

final depthestimation

iterativeoptimization

fi l d thoptimization

init. point cloud

Reconstruction

camera estimationshape alignment

p g

deform shapes

re-estimate depth

matching in 3D

Similar shape extractioninit. correspondence

Experimental Results

Pipeline: utilizes network of similar (not identical) 3D objects to automatically reconstruct the depth information of a single image object.

Input Image Similar Models Depth Recovery Kinect Scan

principle space

principle space

eige

n va

lue

eige

n va

lue

Motivation: There are many more images (estimated in the trillions) than 3D models (estimated in the tens of millions). Estimating depth information on these images enables us to connect them with 3D shapes, allowing us to transfer complementary information back and forth.

Key Idea: Shape deformation subspace can be learned from a network of shapes and then used to attribute a single image with depth.

Shape Deformation Subspace: We learn a deformation model of each shape by its optimized deformations to other shapes. This deformation model serves as the regularizer for local shape space around each shape and is enforced in the reconstruction stage.its optimized deformations to other shapes. This deformation model serves as the regularizer for local shape space around each shape and is enforced in the reconstruction stage.shape space around each shape and is enforced in the reconstruction stage.shape space around each shape and is enforced in the reconstruction stage.

Effect of Deformation Priors: The top/bottom rows show the results without/with the prior for a circular symmetric cup.

Applications

Depth based Relighting

original

original-30 -15

15 30

-30 -15 15 30

Novel View Synthesize