High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron...
-
Upload
elissa-hanby -
Category
Documents
-
view
215 -
download
0
Transcript of High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron...
![Page 1: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/1.jpg)
High-Resolution Three-Dimensional Sensing of Fast Deforming Objects
Philip FongFlorian Buron
Stanford UniversityThis work supported by:
![Page 2: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/2.jpg)
Motivations
• Considerable research on sensing 3D geometry of objects
• Focused on rigid objects and static scenes
• Moving and deforming objects found in many applications
![Page 3: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/3.jpg)
Motivational Applications
• Navigation in dynamic environments
• Object recognition• Human tissue
modeling for surgical simulation and planning
• Robotic manipulationof fabric and rope
![Page 4: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/4.jpg)
Goal
• Generate rangemaps (2.5D) from single images– No temporal coherence assumption– No restriction on motion
• Use commercially available hardware– Minimize cost
![Page 5: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/5.jpg)
Existing 3D Sensing Methods
• Time of flight– Lidar, radar, sonar, shuttered light pulse
• Triangulation– Laser stripe scanner– Stereo– Structured Light
• Can be implemented with commercial cameras and projectors
• Sinusoids / Moiré gratings (Takeda and Kitoh; Tang and Hung, Sansoni et al)
• Stripe patterns (Koninckx, Griesser, and Van Gool; Zhang, Curless, and Seitz; Caspi, Kiryati, and Shamir; Liu, Mu, and Fang)
![Page 6: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/6.jpg)
Limitations of Existing Methods
• Requires scene be rigid and not moving– Laser scanner
• Requires non-repeating texture– Stereo vision
• Applied to deforming cloth (Pritchard and Heidrich)
• Requires known scene topology and known anchor points– Sinusoids / Moiré gratings
• Requires multiple frames– Restricts movement– Spacetime Stereo (Davis, Ramamoothi, and Rusinkiewicz)– Stripes
• In single frames, spatial resolution does not scale due to fixed number of encodings
![Page 7: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/7.jpg)
System Overview
• Idea: Combine colored stripes with sinusoid pattern
• Use sinusoidal pattern to produce dense rangemap
• Colored stripe transitions give sparse absolute depths– Use to generate anchor
points
![Page 8: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/8.jpg)
System Geometry
•Camera projection center at (0,0,0)•Projector at (px, py, pz)•Parallel optical axes•Pinhole projection model for camera and projector
ZW
YW
XW
ZP
YP
XP
px
pz
py
Camera/World Frame
Projector Frame
XPI
YPI
XCI
YCI
Target Object(s)
![Page 9: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/9.jpg)
System Overview
Image
Demodulate Segment
PhaseUnwrap
Find colortransitions
Labelcolors Label
transitionsGenerateguesses
Rangemap
![Page 10: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/10.jpg)
Depth from Sinusoid
• Projected sinusoid:
• Camera sees deformed sinusoid:
• Demodulate to get wrapped phase (Tang and Hung):
CyAyxv )cos(),( PIPIPI
ZWCICIW
YWCI
ZW
CICI
CICICICICI
),(),(
),(cos),(
pyxz
pypyx
yxyAyxv
))cos(),(sin(2arctanw
![Page 11: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/11.jpg)
Segmenting
• Phase unwrapping assumes θ changes by less than 2π between pixels
• Segment image into regions based on phase variance (Ghiglia and Pritt) using snakes
![Page 12: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/12.jpg)
Labeling Colors
• Label and score pixels with colors using Bayesian classifier
• PDFs of colors in hue space are approximated with a gaussian distribution
![Page 13: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/13.jpg)
Labeling Color Transitions
• Threshold change in hue between pixels along X direction
• Label each detected transition according to the projected pattern– Based on color label
scores in pixel windows to the left and right of transition
– Ignore transitions:• Not consistent with
projected pattern• Over a max width
![Page 14: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/14.jpg)
Generating Guesses
• In projected pattern– Each transition appears only once– Identifies unique vertical plane
• Intersect with ray corresponding to transition location in camera to compute depth
• Use as guesses in phase unwrapping
![Page 15: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/15.jpg)
Phase Unwrapping
• Compute phase gradient assuming no jumps greater than 2π
• Integrate to get θu
• For each region compute k from median of the difference between guesses and θu
• Compute rangemap from θ
kyxyx 2),(),( u
![Page 16: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/16.jpg)
Results: Moving Speaker
• 0.7mm (0.1%) RMS error compared to Cyberware 3030MS laser scanner
-40 -20 0 20 40 60 80
90
100
110
120
130
140
150 Speaker Cross-section
X (mm)
De
pth
(m
m)
Our Technique Cyberware 3030MS
![Page 17: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/17.jpg)
Results: Deforming Foam
• Simple deformable object consisting of two types of foam
• System works in presence of color variation
![Page 18: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/18.jpg)
Results: Flag Waving
![Page 19: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/19.jpg)
Advantages
• Spatial resolution scales with camera and projector resolution
• Temporal resolution scales with camera speed– Not limited by projector speed– Complex projector could be eliminated for
static patterns
![Page 20: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/20.jpg)
Limitations
• Each segmented region must contain at least one recognized color transition
• Objects with many saturated colors are hard to sense– Mitigated by choosing right set of colors in
pattern
• Projected pattern must be bright enough to be seen– Difficult to achieve outdoors
![Page 21: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/21.jpg)
Conclusions / Results
• Combined sinusoidal and colored stripe pattern is effective
• Produces good quality dense range maps of moving and deforming objects– Works in presence of color variation– Works in presence of fast motion and large
deformations
![Page 22: High-Resolution Three- Dimensional Sensing of Fast Deforming Objects Philip Fong Florian Buron Stanford University This work supported by:](https://reader036.fdocuments.in/reader036/viewer/2022062515/56649ca35503460f9496407f/html5/thumbnails/22.jpg)
Questions?
More at:http://www.stanford.edu/~fongpwf/research.html