Registration of Colored 3D Point Clouds with a Kernel...
Transcript of Registration of Colored 3D Point Clouds with a Kernel...
![Page 1: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/1.jpg)
Registration of Colored 3D Point Clouds with a Kernel-based Extension to the
Normal Distributions Transform
Benjamin Huhle1, Martin Magnusson2, Wolfgang Straßer1, Achim J. Lilienthal2
1 WSI/GRIS, University of Tübingen, Germany2 AASS, Dept. of Technology, Örebro University,
Sweden
05/23/2008 ICRA '08, Pasadena
Benjamin Huhle - WSI/GRIS, Tübingen
![Page 2: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/2.jpg)
Motivation
2
• point cloud registration for Localization & Mapping
• Problems– geometric features
(structure) required– small field-of-view– noise
• additional color data available!
![Page 3: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/3.jpg)
Outline of the talk
3
• Related Work– Normal Distributions Transform (NDT)– Vision-aided registration (SIFT-Features)– Combined approach (SIFT-Features+NDT)
• Color-NDT– straightforward approach fails– Kernel-based Color-NDT
• Experiments– mobile robot with time-of-flight camera
![Page 4: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/4.jpg)
Normal Distributions Transform (NDT)
4
• Biber & Straßer, 2003– cell grid– approximate point
distributions– multiple overlapping
grids– optimization using
analytical (2nd order) derivatives from: Biber, 2003
![Page 5: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/5.jpg)
3D Normal Distributions Transform
5
• in 3D
• comparison with ICP: Magnusson et al., 2007
![Page 6: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/6.jpg)
Vision-Aided RegistrationAndreasson & Lilienthal, 2007
6
• robust registration with image features (SIFT)– feature detection in images– lookup of 3D coordinates
• challenges:– noise– dynamic environments from: Andreasson et al., 2007
![Page 7: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/7.jpg)
Combined Energy ApproachHuhle, Jenke & Straßer, 2008
7
• Sum of NDT score and feature distances
• must favor features (small )
![Page 8: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/8.jpg)
Ad-hoc approach to using color with NDT
8
• colored point cloud: [x,y,z,r,g,b]• 6D color–space distribution
toy example:• 2D position• 1 color-
dimension (hue)
• model is 3D normal distribution
![Page 9: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/9.jpg)
Single-Mode Color–Space Distribution
9
• conditional distributions of 2 test-points
![Page 10: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/10.jpg)
Single-Mode Color–Space Distribution
10
• ... for a test-point with different color
![Page 11: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/11.jpg)
Kernel-Based Color-NDT
11
• Gaussian mixture-model in color-space (EM-Algorithm)
• components are weighting kernels for point distributions
• use 3 kernels
• for each kernel: compute spatial Normal Distribution using– weighted mean– weighted covariance
![Page 12: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/12.jpg)
Toy Example revisited
12
• perfect match of model and data
![Page 13: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/13.jpg)
Toy Example revisited
13
• ... test point with different color
![Page 14: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/14.jpg)
Registration with Color-NDT
14
• optimize score (mixture model of normal distributions)
• Newton's method• translation + rotation vector (6D parameters)
![Page 15: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/15.jpg)
Experiments
15
• Time-of-flight camera – PMDTec 19k– 160x120 pixels– 30° fov– significant noise level
• additional color camera
![Page 16: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/16.jpg)
Results
16
• 21 incrementally registered frames• odometry as initial poses
![Page 17: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/17.jpg)
Results using combined approach
17
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
![Page 18: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/18.jpg)
Results using combined approach
18
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
![Page 19: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/19.jpg)
Results using combined approach
19
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
![Page 20: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/20.jpg)
Results using combined approach
20
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
![Page 21: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/21.jpg)
Conclusion
21
• Color-NDT:– more robust/stable– more weight on Color-NDT score in combined
approach– can fix inaccuracies of SIFT registration
• towards registration of low-end sensor data with integrated use of color and depth data
![Page 22: Registration of Colored 3D Point Clouds with a Kernel ...130.243.105.49/Research/Learning/publications/2008/... · Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ](https://reader036.fdocuments.in/reader036/viewer/2022081406/5f15c48ba424b941d65cc954/html5/thumbnails/22.jpg)
Thank you for your attention!
22
• Peter Biber and Wolfgang Straßer. The Normal Distributions Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2743–2748, 2003
• Martin Magnusson, Achim Lilienthal and Tom Duckett, Scan Registration for Autonomous Mining Vehicles Using 3D-NDT. Journal of Field Robotics, 24:10, 2007, pp. 803-827
• Henrik Andreasson and Achim J. Lilienthal, Vision Aided 3D Laser Based Registration. Proceedings of the 3rd European Conference on Mobile Robots (ECMR), 2007
• Benjamin Huhle, Philipp Jenke and Wolfgang Straßer. On-the-Fly Scene Acquisition with a Handy Multi-Sensor System. Int. J. of Intelligent Systems Technologies and Applications, to appear, 2008