Data-Driven Shape Analysis --- Geometry...
Transcript of Data-Driven Shape Analysis --- Geometry...
![Page 1: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/1.jpg)
Data-Driven Shape Analysis--- Geometry Reconstruction
Qi-xing HuangStanford University
![Page 2: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/2.jpg)
Geometry Reconstruction
![Page 3: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/3.jpg)
• Occlusion
• Material
• Accuracy/Resolution/Range
• Speed
Issues
![Page 4: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/4.jpg)
Data-Driven Reconstruction
Combine data + priors (from existing shapes)
![Page 5: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/5.jpg)
• Structure-aware
– Data from the same shape (enforced by structures)
– Rich literature in the computer vision community
• Global priors
– Find similar shapes
Two topics
![Page 6: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/6.jpg)
Structure-aware
![Page 7: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/7.jpg)
Context-based Surface Completion [SIG’04]
![Page 8: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/8.jpg)
Approach
![Page 9: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/9.jpg)
Symmetry aware [SIG’08]
![Page 10: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/10.jpg)
Symmetry aware [SIG’10]
![Page 11: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/11.jpg)
Symmetry aware [SIG’ 10]
![Page 12: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/12.jpg)
Algorithm Steps
Detection Registration Consolidation Filtering/sampling
Mapping
![Page 13: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/13.jpg)
Semi-automatic Repetition Detection
• Cluster descriptors
• Descriptors configuration
from user query
• Automatic partial
matching [Bokeloh et al. 2009]
![Page 14: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/14.jpg)
Algorithm Steps
Detection Registration Consolidation Filtering/sampling
Mapping
![Page 15: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/15.jpg)
Segmentation and Registration
• Coarse alignment using ICP
– Based on planes and edges (RANSAC)
– Weighed by plane confidence
• Estimate plane confidence
– Area
– Homogeneity
– Point anisotropy
0 1
![Page 16: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/16.jpg)
Algorithm Steps
Detection Registration Consolidation Filtering/sampling
Mapping
![Page 17: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/17.jpg)
Consolidation
• Off-plane: work on planes
– Cluster planes
– Choose representative planes
– Project points to
representatives
• In-plane: work on edges in
each plane
![Page 18: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/18.jpg)
Clustering
Consolidation
Weighted L1 median consolidation
![Page 19: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/19.jpg)
Algorithm Steps
Detection Registration Consolidation Filtering/sampling
Mapping
![Page 20: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/20.jpg)
Filtering of lines based on major axis directions
Partition plane into rectangles
Classify to inliers, outliers
Remove outliers, upsample
Filtering and Upsampling
![Page 21: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/21.jpg)
Results: cylinders consolidation
![Page 22: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/22.jpg)
Combining Images and Shapes [ICCV’11]
Detecting symmetries in the image domain
![Page 23: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/23.jpg)
Structure from motion [TOG’14]
Detect symmetries in 3D
Symmetries are enforced in SFM
Coupled Structure-from-Motion and 3D Symmetry Detection for Urban Facades, TOG’14
![Page 24: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/24.jpg)
Global Shape Prior
![Page 25: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/25.jpg)
Example-Based Scan Completion
Example-Based 3D Scan Completion, SGP’05
![Page 26: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/26.jpg)
Examples
Final Model
Context Models
Deformed Models
![Page 27: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/27.jpg)
Search-Classify [TOG’12]
A Search-Classify Approach for Cluttered Indoor Scene Understanding, TOG’12
Two interleaved problems
What are the objects?
Where are the objects?
![Page 28: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/28.jpg)
• Search
–Propagate /
accumulate
patches
• Classify
–Query
classifier to
detect object
Approach
![Page 29: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/29.jpg)
Overview
Training
Search-Classify
![Page 30: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/30.jpg)
Point-cloud Features– Height-size ratio of BBox
– Aspect ratio of each layer
– Bottom-top, mid-top size ratio
– Change in COM along horizontal slabs
Bh
BdBw
![Page 31: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/31.jpg)
Search-Classify• Starts from seeds
– Random patch triplets
– Remove seeds with low confidence
• Accumulating neighbor patches
– Highest classification confidence
• Stop condition
– Steep decrease in classification confidence
0.65 0.92 0.93 0.88
Seed
![Page 32: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/32.jpg)
Refinement via template fitting• Segmented - classified objects problems
– Overlap, outliers, ambiguities etc.
• Refinement
– Outliers = patches with large distance
![Page 33: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/33.jpg)
Results
![Page 34: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/34.jpg)
Contextual information is no considered
![Page 35: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/35.jpg)
Part-based Shape Reconstruction [TOG’12]
No suitable model!shape retrieval
Structure Recovery by Part Assembly TOG’12
![Page 36: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/36.jpg)
Recover the structure by part assembly
![Page 37: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/37.jpg)
Algorithm Overview
Candidate Parts Selection Structure Composition Part Conjoining
……
![Page 38: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/38.jpg)
Algorithm Overview
Candidate Parts Selection Structure Composition Part Conjoining
……
![Page 39: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/39.jpg)
Algorithm Overview
Candidate Parts Selection Structure Composition Part Conjoining
……
![Page 40: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/40.jpg)
Algorithm Overview
Candidate Parts Selection Structure Composition Part Conjoining
……
![Page 41: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/41.jpg)
Results: Chairs• 70 repository models, 11 part categories
![Page 42: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/42.jpg)
Results: Tables• 61 repository models, 4 part categories
![Page 43: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/43.jpg)
Results: Bicycles
• 38 repository models, 9 part categories
![Page 44: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/44.jpg)
Results: Airplanes
• 70 repository models, 6 part categories
![Page 45: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/45.jpg)
Future directions
![Page 46: Data-Driven Shape Analysis --- Geometry Reconstructiongraphics.stanford.edu/courses/cs468-14-spring/... · •Descriptors configuration from user query •Automatic partial matching](https://reader034.fdocuments.in/reader034/viewer/2022043008/5f9624d28b81092697403102/html5/thumbnails/46.jpg)
• Grammar-based
– Understand the variability of a class of shapes [Hao et al. 14]
• Scene reconstruction/Understanding
• Data-driven dynamic geometry reconstruction
Future Directions