Estimating 3D Attributes of Images from Shape Collections · Estimating 3D Attributes of Images...
27
Estimating 3D Attributes of Images from Shape Collections Hao Su, Qixing Huang, Yangyan Li, Niloy Mitra, Leo Guibas
Transcript of Estimating 3D Attributes of Images from Shape Collections · Estimating 3D Attributes of Images...
Estimating 3D Attributes of Images from
Shape Collections
Hao Su, Qixing Huang, Yangyan Li,
Niloy Mitra, Leo Guibas
Cross-domain Network
3D SHAPES
TEXT IMAGES
• image annotation
• image search
• image collage
• …
• shape recognition ?
Image world Shape world
Very big: Trillions Big: Tens of millions
1012
1010
108
106
104
Images 3D Models
2007
2014
Image world Shape world
Very big: Trillions Big: Tens of millions
Well annotated Sparse annotation
Image world Shape world
Very big: Trillions Big: Tens of millions
Rich labels Sparse labels
2D 3D
Information Transportation
Automatic shape tagging
Image world Shape world
Labels
Information Transportation
Pose estimation
Cam
params
Image world Shape world
Liebelt et al., Savarase et al, …
Information Transportation
Depth transfer
3D
Image world Shape world
Related work: 3D reconstruction
Multi-view geometry
– Structure from Motion
– Binocular Disparity
Single image
reconstruction
– Automatic Photo
Pop-up,
– Make 3D
– …Ill-defined problem
What’s PRIOR knowledge
to use?
Key idea
Non-rigid Shape Deformation
Data Driven by Shape Collection
Shape space
Image space
projection
A high-level overview
Shape space
Image space
projection
Pose Estimation
Build initial registration
Refine depth estimation
Pipeline
Pair-wise image-shape
matching
Data-driven initial viewpoint
estimation
– Align shapes using shape-
network
Pose EstimationBuild initial registration
Refine depth estimation
Matching landmark points in 2D space
Pose EstimationBuild initial registration
Refine depth estimation
Single shape Multi-shapes
Pose EstimationBuild initial registration
Refine depth estimation
Shape space
LEARNING LOCAL REGULAR
DEFORMATION SPACE
Diagnosis: effect of deformation prior
without prior
with prior
View 1 View 2 View 3
Learning deformation prior
Diagnosis: effect of deformation prior
without prior
with prior
View 1 View 2 View 3
Strong Prior Knowledge:
Local Shape Deformation Subspace
EXPERIMENT RESULTS
Thank you!
