Context-based Surface Completion Andrei Sharf, Marc Alexa, Daniel Cohen-Or.
-
Upload
kelley-owens -
Category
Documents
-
view
217 -
download
0
Transcript of Context-based Surface Completion Andrei Sharf, Marc Alexa, Daniel Cohen-Or.
Context-based Context-based Surface Surface
CompletionCompletion
Andrei Sharf, Marc Alexa, Daniel Cohen-OrAndrei Sharf, Marc Alexa, Daniel Cohen-Or
IntroductionIntroductionIntroductionIntroduction
Holes in models:• Imperfect range scanned
data:– Complex objects with non-visible
regions.– Misalignment of multiple-views
depth image scans.– Material reflections.
• Surface editing operations.
MotivationMotivationMotivationMotivation
Smooth filling is sufficient:• Small holes.
• Smooth surfaces.
Smooth filling is insufficient :• Surfaces containing fine
geometric detail.
• Topology of hole is more complex than a disk.
Context-based Completion Context-based Completion Context-based Completion Context-based Completion
Complete the missing region with patches that conform with its context
Smooth Context-based
MethodMethodMethodMethod
Import patches with matching context from the surface itself :
• Analyze surface characteristics.
• Find best matching patch.
• Fit imported patch to boundary.
Related WorkRelated WorkRelated WorkRelated Work
• Example-based image completion:[Drori et al. 2003; Jia and tang 2003; Sun et al. 2003;]
• Texture synthesis:[Efros and Leung 1999; Efros and Freeman 2001;Wei and Levoy 2000; Ying et al. 2001;]
• Smooth surface completion:[Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003;]
• Model-based surface reconstruction [Savchenko et al. 2002]
• Curve analogies [Hertzmann et al. 2002]
Drori et al. 03
Liepa 03
Wei and Levoi 00
Hertzmann et al. 02
Moving to 3D SurfacesMoving to 3D SurfacesMoving to 3D SurfacesMoving to 3D Surfaces
Images have a regular spatial structure domain.
Problems in 3D:• Topology and geometry of missing region.
• Fitting a patch to the boundary of the missing region.
• Definition of similarity of shapes.
• Definition of a surface patch.
AlgorithmAlgorithmAlgorithmAlgorithm
• Given an incomplete shape• Create initial spatial subdivision• For each cell
– Compute a local shape representation.– Compute a shape signature.
• For each empty cell:– Find matching nonempty cell ω’.– Copy patch of ω’ into ω.
• Subdivide cells and repeat• Until completed region matches
its neighborhood.
AlgorithmAlgorithmAlgorithmAlgorithm
• Given an incomplete shape• Create initial spatial subdivision• For each cell
– Compute a local shape representation.– Compute a shape signature.
• For each empty cell:– Find matching nonempty cell ω’.– Copy patch of ω’ into ω.
• Subdivide cells and repeat• Until completed region matches
its neighborhood.
Shape RepresentationShape RepresentationShape RepresentationShape Representation
• Shape sampled point-set– Range scan output– Easy to merge
• Octree hierarchy on top of the point set
• Implicit surface approximation by fitting polynomials [Ohtake et al. 2003]:– General quadric:– Bivariate quadratic:– Edge or corner fitting
( ) t tQ x x Ax b x c 2 2( ) ( )Q x w Au Buv Cv Du Ev F
,i ip n
0Q x
Shape AnalysisShape AnalysisShape AnalysisShape Analysis
A local signature of a cell consists:• Implicit shape characteristics:
– Signed distance– Normal variation
• Detail amount:– Depth in octree
AlgorithmAlgorithmAlgorithmAlgorithm
• Given an incomplete shape• Create initial spatial subdivision• For each cell
– Compute a local shape representation.– Compute a shape signature.
• For each empty cell:– Find matching nonempty cell ω’.– Copy patch of ω’ into ω.
• Subdivide cells and repeat• Until completed region matches
its neighborhood.
Missing RegionMissing RegionMissing RegionMissing Region
Automatic identification:• Intersect shape approximation with cells.
• Empty cell that intersects surface is part of missing region.
Empty cell definition can alter due to refinement of missing region.
MatchingMatchingMatchingMatching
Distance metrics:• dc : Signature distance inside cells.
• da : Signature distance of adjacent cells.
• dl : Difference in amount of detail.
i c i a l ld( , ') = w d ( , ')+(1-w )d ( , ')+w d ( , ') Find most similar non-empty cell:
Matching-Candidate SetMatching-Candidate SetMatching-Candidate SetMatching-Candidate Set
• Non-empty cells of the same size
• Symmetry rotations of cells(/2, , ...)
• Rotation of all point-set with some angle (/4, /3, ...)
Transferring PointsTransferring PointsTransferring PointsTransferring Points
ICP:• Copy points into empty cell.
• Find closest point correspondence
• Align points rigidly
• Align points non-rigidly using polynomial form.
AlgorithmAlgorithmAlgorithmAlgorithm
• Given an incomplete shape• Create initial spatial subdivision• For each cell
– Compute a local shape representation.– Compute a shape signature.
• For each empty cell:– Find matching nonempty cell ω’.– Copy patch of ω’ into ω.
• Subdivide cells and repeat• Until completed region matches
its neighborhood.
David’s HairDavid’s HairDavid’s HairDavid’s Hair
Original Down-sampled
Smooth completion
Context-based completion
Completion ProcessCompletion ProcessCompletion ProcessCompletion Process
Original
Initial approximation
Final result
Manual Editing of Bunny ModelManual Editing of Bunny ModelManual Editing of Bunny ModelManual Editing of Bunny Model
Manual Editing of Knot ModelManual Editing of Knot ModelManual Editing of Knot ModelManual Editing of Knot Model
Scan of “Youth” Statue (Rear View)Scan of “Youth” Statue (Rear View)Scan of “Youth” Statue (Rear View)Scan of “Youth” Statue (Rear View)
Original Smooth Result
Sample Rate vs. Detail Sample Rate vs. Detail FrequencyFrequencySample Rate vs. Detail Sample Rate vs. Detail FrequencyFrequency
SummarySummarySummarySummary
A fully automatic method to complete a missing region in a surface from its context.
• Completed patches geometrically conform with neighborhood.
• Incremental scale-space framework for finer approximation of the unknown region.
Future WorkFuture WorkFuture WorkFuture Work
• Explore with other spatial hierarchies.
• Enlarge the search space of examples by building a class based example set.
• Couple method with image completion methods for texture completion.
• Rotation and translation invariant signature
Acknowledgments:– David Levin and Olga Sorkine– Israel science foundation; Israeli ministry of science– Digital Michelangelo project 3D model repository; Stanford 3D
scanning repository; Darmstadt university of technology; Imager computer graphics laboratory of the university of British Columbia