Interactive 3D Modeling of Indoor Environments with a Consumer Depth Camera
33
Interactive 3D Modeling of Indoor Environments with a Consumer Depth Camera Hao Du 3 , Peter Henry 1 , Xiaofeng Ren 2 , Marvin Cheng 1 , Dan B Goldman 4 , Steven M. Seitz 1 , Dieter Fox 1,2 1 University of Washington: Computer Science & Engineering 2 Intel Labs Seattle (now ISTC at UW) 3 Google 4 Adobe Systems
description
Interactive 3D Modeling of Indoor Environments with a Consumer Depth Camera. Hao Du 3 , Peter Henry 1 , Xiaofeng Ren 2 , Marvin Cheng 1 , Dan B Goldman 4 , Steven M. Seitz 1 , Dieter Fox 1,2 1 University of Washington: Computer Science & Engineering 2 Intel Labs Seattle (now ISTC at UW) - PowerPoint PPT Presentation
Transcript of Interactive 3D Modeling of Indoor Environments with a Consumer Depth Camera
Interactive 3D Modeling of Indoor Environments with
a Consumer Depth Camera
Hao Du3, Peter Henry1, Xiaofeng Ren2, Marvin Cheng1,Dan B Goldman4, Steven M. Seitz1, Dieter Fox1,2
1University of Washington: Computer Science & Engineering2Intel Labs Seattle (now ISTC at UW)
3Google4Adobe Systems
Outline1. Introduce RGB-D (Red, Green, Blue + Depth)
mapping with a Kinect camera2. Show how an interactive mapping system
improves usability and results3. Demonstrate potential applications to
localization, measurement, and interactive visualization
The Kinect
Guinness record: “fastest selling consumer device”
$150, more than 10 million sold as of March 2011
OpenNI and Kinect SDK allow for use by developers
RGB-D CameraRed Green Blue
Depth
[PrimeSense.com]
RGB-D Data
Interactive Mapping
RGB-D Frames
Local Alignment (RANSAC) Global Alignment
Loop ClosureDetection
Update Model
RGB-D Mapping*RGB-D frames are recordedThe sequence is then processed to obtain the
map
*RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments. Henry et al. International Symposium on Experimental Robotics (ISER), 2010.
Frame Matching with RANSAC(Random Sample Consensus)
Visual features (from image) in 3D (from depth)Figure out how the camera moved by matching
these features to the previous frame
Limitations of PreviousRGB-D Mapping
Limited to offline processing after recordingEven a small number of missing alignments
result in a uselessly disconnected mapExpert knowledge of the system is required to
get good resultsNo feedback on coverage of the environment
Interactive 3D Mapping
RGB-D Frames
Local Alignment (RANSAC) Global Alignment
Loop ClosureDetection
Update Model
RGB-D Frames
Failure Detection
Local Alignment (RANSAC) Global Alignment
Loop ClosureDetection
CompletenessAssessment
User Moves and Resumes Mapping
User Verifies and Adds Loop Closures
User Acquires Additional Frames
Visualization
User Controls Viewpoint
Update Model
Suggest Places to Visit
InteractiveRGB-D Mapping
User Interface
Examining Model Completeness
Completeness Hints
Interactive System Evaluation
4 users (2 novice, 2 experienced) were instructed to map three walls of a room both with and without the interactive system
All experienced failures without the interactive system (after a mean count of 40.75 frames)
The proposed interactive system caught 26 potential failures allowing all users to interactively go back and create accurate maps
Non-interactive
Interactive
Final Position Error (m)
1.13 0.08
Benefits of Interactive 3D Mapping
RobustnessReal-time feedback of mapping status and failuresRewind / Recover / ResumeDelete frames (if a person walks in front of you,
etc.)Remove false-positive loop closuresPropose additional loop closures
CompletenessSee the map as it is builtShow unexplored areas
Large Scale Map
Espresso Room Model
Applications1. Localization2. Measuring Dimensions3. Interactive Visualization
Localization
LocalizationCentimeter level accuracy within an existing
mapProvides full 3D translation and rotationRequires no instrumentation of the environmentAn alternative to RFID / 802.11 / GSM /
PowerLine based localization in GPS denied environments
Measuring Dimensions
Online real estate salesVirtual remodelingOnline furniture shoppingVirtual walkthrough for educationRemote telepresense
Interactive Visualization
Interactive gesture controlled visualization
ConclusionIncorporating user interaction into real-time 3D
mapping allows non-experts to create accurate maps using RGB-D cameras (such as the Kinect)
These maps have applications to localization, measurement, and visualization
Making this system publicly available will allow for a proliferation of 3D mapping and the potential for new context-aware applications
