High resolution 3D seismic undershooting acquisition over ...
Real-Time 3D Model Acquisition
-
Upload
vladimir-turner -
Category
Documents
-
view
22 -
download
1
description
Transcript of Real-Time 3D Model Acquisition
Real-Time 3D Model Real-Time 3D Model AcquisitionAcquisition
Szymon RusinkiewiczSzymon Rusinkiewicz
Olaf Hall-HoltOlaf Hall-Holt
Marc LevoyMarc Levoy
Princeton UniversityPrinceton University
Stanford UniversityStanford University
3D Scanning3D Scanning
Possible Research GoalsPossible Research Goals
• Low noiseLow noise
• Guaranteed high accuracyGuaranteed high accuracy
• High speedHigh speed
• Low costLow cost
• Automatic operationAutomatic operation
• No holesNo holes
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
View PlanningView PlanningView PlanningView Planning
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignmentView PlanningView PlanningView PlanningView Planning
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMerging
View PlanningView PlanningView PlanningView Planning
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
3D Model Acquisition Pipeline3D Model Acquisition Pipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
3D Model Acquisition Difficulties3D Model Acquisition Difficulties
• Much (often most) time spent on “last Much (often most) time spent on “last 20%”20%”
• Pipeline not optimized for hole-fillingPipeline not optimized for hole-filling
• Not sufficient just to speed up scanner –Not sufficient just to speed up scanner –must design pipeline for fast feedbackmust design pipeline for fast feedback
Real-Time 3D Model AcquisitionReal-Time 3D Model Acquisition
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
HumanHuman
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Challenge:Challenge:Real TimeReal Time
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Part I:Part I:Structured-LightStructured-Light
TriangulationTriangulation
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Part II:Part II:Fast ICPFast ICP
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Part III:Part III:Voxel GridVoxel Grid
TriangulationTriangulation
• Project laser stripe onto objectProject laser stripe onto object
ObjectObject
LaserLaser
CameraCameraCameraCamera
CameraCameraCameraCamera
TriangulationTriangulation
• Depth from ray-plane triangulationDepth from ray-plane triangulation
LaserLaser
(x,y(x,y))
ObjectObject
TriangulationTriangulation
• Faster acquisition: project multiple Faster acquisition: project multiple stripesstripes
• Correspondence problem: which stripeCorrespondence problem: which stripeis which?is which?
Continuum of Triangulation Continuum of Triangulation MethodsMethods
Slow, robustSlow, robust Fast, fragileFast, fragile
Multi-stripeMulti-stripeMulti-frameMulti-frame
Single-frameSingle-frameSingle-stripeSingle-stripe
Time-Coded Light PatternsTime-Coded Light Patterns
• Assign each stripe a unique illumination Assign each stripe a unique illumination codecodeover time [Posdamer 82]over time [Posdamer 82]
SpaceSpace
TimeTime
Illumination history = (WB),(BW),(WB)Illumination history = (WB),(BW),(WB)Illumination history = (WB),(BW),(WB)Illumination history = (WB),(BW),(WB)
CodeCodeCodeCode
Codes for Moving ScenesCodes for Moving Scenes
• Assign time codesAssign time codesto stripe to stripe boundariesboundaries
• Perform frame-to-framePerform frame-to-frametracking of correspondingtracking of correspondingboundariesboundaries– Propagate illumination historyPropagate illumination history
[Hall-Holt & Rusinkiewicz, ICCV 2001][Hall-Holt & Rusinkiewicz, ICCV 2001]
Designing a CodeDesigning a Code
• Want many “features” to track:Want many “features” to track:lots of black/white edges at each framelots of black/white edges at each frame
• Try to minimize ghosts – WW or BB Try to minimize ghosts – WW or BB “boundaries” that can’t be seen directly“boundaries” that can’t be seen directly
Designing a CodeDesigning a Code
0011001111101110
10111011 01100110
01000100 10011001
0001000100010001 1100110011001100
00000000 11011101
10101010 01110111
10001000010101011111111111111111 0010001000100010
[Hall-Holt & Rusinkiewicz, ICCV 2001][Hall-Holt & Rusinkiewicz, ICCV 2001]
ImplementationImplementation
• Pipeline:Pipeline:
• DLP projector illuminates scene @ 60 DLP projector illuminates scene @ 60 Hz.Hz.
• Synchronized NTSC camera captures Synchronized NTSC camera captures videovideo
• Pipeline returns range images @ 60 Hz.Pipeline returns range images @ 60 Hz.
ProjectProjectCodeCode
ProjectProjectCodeCode
CaptureCaptureImagesImagesCaptureCaptureImagesImages
FindFindBoundariesBoundaries
FindFindBoundariesBoundaries
MatchMatchBoundariesBoundaries
MatchMatchBoundariesBoundaries DecodeDecodeDecodeDecode ComputeCompute
RangeRangeComputeCompute
RangeRange
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Part II:Part II:Fast ICPFast ICP
Aligning 3D DataAligning 3D Data
• ICP (Iterative Closest Points): for each ICP (Iterative Closest Points): for each point on one scan, minimize distance to point on one scan, minimize distance to closest point on other scan…closest point on other scan…
Aligning 3D DataAligning 3D Data
• … … and iterate to find alignmentand iterate to find alignment– Iterated Closest Points (ICP) [Besl & McKay Iterated Closest Points (ICP) [Besl & McKay
92]92]
ICP in the Real-Time PipelineICP in the Real-Time Pipeline
• Potential problem with ICP: local minimaPotential problem with ICP: local minima– In this pipeline, scans close togetherIn this pipeline, scans close together
– Very likely to converge to correct (global) Very likely to converge to correct (global) minimumminimum
• Basic ICP algorithm too slow (~ seconds)Basic ICP algorithm too slow (~ seconds)– Point-to-plane minimizationPoint-to-plane minimization
– Projection-based matchingProjection-based matching
– With these tweaks, running time ~ millisecondsWith these tweaks, running time ~ milliseconds
[Rusinkiewicz & Levoy, 3DIM 2001][Rusinkiewicz & Levoy, 3DIM 2001]
Real-Time 3D Model Acquisition Real-Time 3D Model Acquisition PipelinePipeline
3D Scanner3D Scanner3D Scanner3D Scanner
AlignmentAlignmentAlignmentAlignment
MergingMergingMergingMergingDone?Done?Done?Done?
View PlanningView PlanningView PlanningView Planning
DisplayDisplayDisplayDisplay
Part III:Part III:Voxel GridVoxel Grid
Merging and RenderingMerging and Rendering
• Goal: visualize the model well enoughGoal: visualize the model well enoughto be able to see holesto be able to see holes
• Cannot display all the scanned data – Cannot display all the scanned data – accumulates linearly with timeaccumulates linearly with time
• Standard high-quality merging methods:Standard high-quality merging methods:processing time ~ 1 minute per scanprocessing time ~ 1 minute per scan
Merging and RenderingMerging and Rendering
Merging and RenderingMerging and Rendering
Merging and RenderingMerging and Rendering
Merging and RenderingMerging and Rendering
++
Merging and RenderingMerging and Rendering
• Point rendering, using accumulated normals for lightingPoint rendering, using accumulated normals for lighting
Example: PhotographExample: Photograph
18 cm.18 cm.
ResultResult
PostprocessingPostprocessing
• Real-time displayReal-time display– Quality/speed tradeoffQuality/speed tradeoff
– Goal: let user evaluate coverage, fill holesGoal: let user evaluate coverage, fill holes
• Offline postprocessing for high-quality Offline postprocessing for high-quality modelsmodels– Global registrationGlobal registration
– High-quality merging (e.g., using VRIP High-quality merging (e.g., using VRIP [Curless 96])[Curless 96])
Postprocessed ModelPostprocessed Model
Recapturing AlignmentRecapturing Alignment
SummarySummary
• 3D model acquisition pipeline optimized 3D model acquisition pipeline optimized for obtaining complete, hole-free modelsfor obtaining complete, hole-free models
• Use human’s time most efficientlyUse human’s time most efficiently
• Pieces of pipeline selected for real-time Pieces of pipeline selected for real-time use:use:– Structured-light scanner for moving objectsStructured-light scanner for moving objects
– Fast ICP variantFast ICP variant
– Simple grid-based merging, point renderingSimple grid-based merging, point rendering
LimitationsLimitations
• Prototype noisier than commercial systemsPrototype noisier than commercial systems– Could be made equivalent with careful engineeringCould be made equivalent with careful engineering
– Ultimate limitations on quality: focus, textureUltimate limitations on quality: focus, texture
• Scan-to-scan ICP not perfect Scan-to-scan ICP not perfect alignment drift alignment drift– Due to noise, miscalibration, degenerate geometryDue to noise, miscalibration, degenerate geometry
– Reduced, but not eliminated, by “anchor scans”Reduced, but not eliminated, by “anchor scans”
– Possibly combine ICP with separate trackersPossibly combine ICP with separate trackers
Future WorkFuture Work
• Faster scanningFaster scanning– Better stripe boundary trackingBetter stripe boundary tracking
– Multiple cameras, projectorsMultiple cameras, projectors
– High-speed cameras, projectorsHigh-speed cameras, projectors
• Application in different contextsApplication in different contexts– Cart- or shoulder-mounted for digitizing Cart- or shoulder-mounted for digitizing
roomsrooms
– Infrared for imperceptibilityInfrared for imperceptibility
AcknowledgmentsAcknowledgments
• Collaborators:Collaborators:– Li-Wei HeLi-Wei He
– James DavisJames Davis
– Lucas PereiraLucas Pereira
– Sean AndersonSean Anderson
• Sponsors:Sponsors:– SonySony
– IntelIntel
– IntervalInterval