PCL :: Features - Point Cloud Librarypointclouds.org/assets/iros2011/features.pdf · 2011-10-06 ·...
Transcript of PCL :: Features - Point Cloud Librarypointclouds.org/assets/iros2011/features.pdf · 2011-10-06 ·...
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
PCL :: FeaturesMichael Dixon and Radu B. Rusu
July 1, 2011
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Outline
1. Introduction
2. Feature Estimation
3. Keypoint Detection
4. Combining Keypoints and Features
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Introduction
In this session, we’ll talk about features and keypoints in PCL.
I Local feature estimationI Surface normal estimationI Point Feature Histogram descriptors
I Keypoint detectionI 3D SIFT keypoint detection
I Global feature estimationI Viewpoint Feature Histogram descriptors
Download the code and data at
http://dev.pointclouds.org/attachments/download/429/IROS-PCL-tutorial-code.zip
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Outline
1. Introduction
2. Feature Estimation
3. Keypoint Detection
4. Combining Keypoints and Features
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local FeaturesWhat is a feature?
What is a feature?
In vision/perception, the word “feature” can mean manydifferent things. In PCL, “feature estimation” means:
I computing a feature vector based on each points localneighborhood
I or sometimes, computing a single feature vector for thewhole cloud
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local FeaturesWhat is a feature?
Feature vectors can be anything from simple surface normals tothe complex feature descriptors need for registration or objectdetection.
Today, we’ll look at a couple of examples:
I Surface normal estimation (NormalEstimation)I Point Feature Histogram estimation (FPFHEstimation)I Viewpoint Feature Histogram estimation
(VFHEstimation)
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local Features (1-2/5)Surface Normal Estimation. Theoretical Aspects: Basic Ingredients
I Given a point cloud with x,y,z 3D point coordinates
I Select each point’s k -nearest neighbors, fit a local plane,and compute the plane normal
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local Features (1-2/5)Surface Normal Estimation. Theoretical Aspects: Basic Ingredients
I Given a point cloud with x,y,z 3D point coordinates
I Select each point’s k -nearest neighbors, fit a local plane,and compute the plane normal
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local Features (3/5)Surface Normal and Curvature Estimation
Cj =∑k
i=1 ξi · (pi − pj)T · (pi − pj), p = 1
k ·∑k
i=1 pi
ξi =
e− d2
iµ2 , pi outlier
1, pi inlier
σp = λ0λ0+λ1+λ2
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Feature estimation basics
I Create the feature estimation objectI (Templated on the input and output point types)
I Set the search method to use for the underlyingneighborhood searches
I Create a k-D Tree and pass it to the feature estimatorI Specify the size of the local neighborhood to use when
computing the featuresI setRadiusSearch (r) or setNearestKSearch (k)
I Set the input pointsI Takes a shared pointer to a pcl::PointCloud
I ComputeI Outputs into a reference to a pcl::PointCloud
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
NormalEstimation example
Estimating surface normals using pcl::NormaLEstimationhttp://docs.pointclouds.org/1.1.0/classpcl_1_1_normal_estimation.html
SurfaceNormalsPtrestimateSurfaceNormals (const PointCloudPtr & input, float radius){
pcl::NormalEstimation<PointT, NormalT> normal_estimation;normal_estimation.setSearchMethod(pcl::KdTreeFLANN<PointT>::Ptr (new pcl::KdTreeFLANN<PointT>));
normal_estimation.setRadiusSearch (radius);
normal_estimation.setInputCloud (input);
SurfaceNormalsPtr normals (new SurfaceNormals);normal_estimation.compute (*normals);
return (normals);}
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
NormalEstimation results
$ ./test_feature_estimation robot.pcd -n 0.03
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Now let’s look at a more complex feature, like...
I Point Feature Histograms (PFH)
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Point Feature Histograms (PFH)A 3D feature descriptor
I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v
f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)
ihist =∑x≤3
x=0
⌊fx ·d
fxmax−fxmin
⌋· dx
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Point Feature Histograms (PFH)A 3D feature descriptor
f0 = 〈v ,nj〉f1 = 〈u, pj − pi〉/||pj − pi ||f2 = ||pj − pi ||f3 = atan(〈w ,nj〉, 〈u,nj〉)
ihist =∑x≤3
x=0
⌊fx ·d
fxmax−fxmin
⌋· dx
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Point Feature Histograms (PFH)A 3D feature descriptor
I For every point pair〈(ps,ns); (pt ,nt)〉, letu = ns, v = (pt−ps)×u, w = u×v
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Point Feature Histograms (PFH)Points lying on different geometric primitives
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
As an example, we’ll compute Fast Point Feature Histgramdescriptors (a faster approximation of PFH)
I Even though FPFH feature descriptors are more complex thansurface normals, the code is very similar
I The main difference: we need to set input normals
I First use pcl::EstimateNormalsI Then pass in a shared pointer to the resulting normals
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
FPHFEstimation example
LocalDescriptorsPtrcomputeLocalDescriptors (const PointCloudPtr & points, const SurfaceNormalsPtr & normals,
float feature_radius){
pcl::FPFHEstimation<PointT, NormalT, LocalDescriptorT> fpfh_estimation;fpfh_estimation.setSearchMethod(pcl::KdTreeFLANN<PointT>::Ptr (new pcl::KdTreeFLANN<PointT>));
fpfh_estimation.setRadiusSearch (feature_radius);fpfh_estimation.setInputCloud (points);
fpfh_estimation.setInputNormals (normals);
LocalDescriptorsPtr local_descriptors (new LocalDescriptors);fpfh_estimation.compute (*local_descriptors);
return (local_descriptors);}
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Outline
1. Introduction
2. Feature Estimation
3. Keypoint Detection
4. Combining Keypoints and Features
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
KeypointsWhat is a keypoint?
What is a keypoint?
I A keypoint (also known as an “interest point”) is simply apoint that has been identified as a relevant in some way.
I Whether any given point is considered a keypoint or notdepends on the keypoint detector in question.
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
KeypointsWhat is a keypoint?
There’s no strict definition for what constitutes a keypointdetector, but a good keypoint detector will find points whichhave the following properties:
I Sparseness: Typically, only a small subset of the points inthe scene are keypoints
I Repeatiblity: If a point was determined to be a keypoint inone point cloud, a keypoint should also be found at thecorresponding location in a similar point cloud. (Suchpoints are often called "stable".)
I Distinctiveness: The area surrounding each keypointshould have a unique shape or appearance that can becaptured by some feature descriptor
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
KeypointsWhy find keypoints?
What are the benefits of keypoints?
I Some features are expensive to compute, and it would beprohibitive to compute them at every point. Keypointsidentify a small number of locations where computingfeature descriptors is likely to be most effective.
I When searching for corresponding points, featurescomputed at non-descriptive points will lead to ambiguousfeature corespondences. By ignoring non-keypoints, onecan reduce error when matching points.
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
KeypointsKeypoint detection in PCL
There are a couple of keypoint detectors in PCL so far.I In this tutorial, we’ll focus on pcl::SIFTKeypoint
I A 3D adaptation of David Lowe’s SIFT keypoint detector
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
SIFT keypoint detection
The SIFT keypoint detector finds local extrema in adifference-of-Gaussians (DoG) scale-space. This scale spaceand its extrema are computed based on several parameters:
I the size of the smallest scaleI the number of times the scale doublesI the number of scales in between each doublingI the minimum absolute DoG value needed to qualify as a
keypoint
Now let’s look at an example of how to compute 3D SIFTkeypoints in PCL
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
SIFTKeypoints example
Computing keypoints on a cloud of pointsPointCloudPtrdetectKeypoints (const PointCloudPtr & points, const SurfaceNormalsPtr & normals,
float min_scale, int nr_octaves, int nr_scales_per_octave, float min_contrast){
pcl::SIFTKeypoint<PointT, pcl::PointWithScale> sift_detect;sift_detect.setSearchMethod(pcl::KdTreeFLANN<PointT>::Ptr (new pcl::KdTreeFLANN<PointT>));
sift_detect.setScales (min_scale, nr_octaves, nr_scales_per_octave);sift_detect.setMinimumContrast (min_contrast);sift_detect.setInputCloud (points);
pcl::PointCloud<pcl::PointWithScale> keypoints_temp;sift_detect.compute (keypoints_temp);
PointCloudPtr keypoints (new PointCloud);pcl::copyPointCloud (keypoints_temp, *keypoints);
return (keypoints);}
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Outline
1. Introduction
2. Feature Estimation
3. Keypoint Detection
4. Combining Keypoints and Features
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Combining keypoints and features
Keypoints and feature descriptors go hand-in-hand.
Let’s look at how to compute features only at a given set ofkeypoints.
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Estimating features at a subset of points
I Create the feature estimation objectI Set the search method to use for the underlying
neighborhood searchesI Specify the size of the local neighborhood to use when
computing the featuresI Set the search surface
I These points are used when searching for the input points’neighbors
I Set the input pointsI These are the points for each feature is computed
I Compute
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Keypoints and features
Computing local feature descriptors for a set of keypointsLocalDescriptorsPtrcomputeLocalDescriptors (const PointCloudPtr & points, const SurfaceNormalsPtr & normals,
const PointCloudPtr & keypoints, float feature_radius){
pcl::FPFHEstimation<PointT, NormalT, LocalDescriptorT> fpfh_estimation;fpfh_estimation.setSearchMethod (pcl::KdTreeFLANN<PointT>::Ptr (new pcl::KdTreeFLANN<PointT>));fpfh_estimation.setRadiusSearch (feature_radius);fpfh_estimation.setSearchSurface (points);fpfh_estimation.setInputNormals (normals);fpfh_estimation.setInputCloud (keypoints);
LocalDescriptorsPtr local_descriptors (new LocalDescriptors);fpfh_estimation.compute (*local_descriptors);
return (local_descriptors);}
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Global feature descriptors
Computing global feature descriptors
A global feature descriptor isI A single descriptor computed over the entire cloud.I Used to compare clouds for object recognition later in this
tutorialAs an example, we’ll be using Viewpoint Feature Histograms
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
VFHEstimation
Computing a VFH feature descriptorGlobalDescriptorsPtrcomputeGlobalDescriptor (const PointCloudPtr & points, const SurfaceNormalsPtr & normals){
pcl::VFHEstimation<PointT, NormalT, GlobalDescriptorT> vfh_estimation;vfh_estimation.setSearchMethod (pcl::KdTreeFLANN<PointT>::Ptr (new pcl::KdTreeFLANN<PointT>));vfh_estimation.setInputCloud (points);vfh_estimation.setInputNormals (normals);
GlobalDescriptorsPtr global_descriptor (new GlobalDescriptors);vfh_estimation.compute (*global_descriptor);
return (global_descriptor);}
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Testing
Use test_feature_estimation to run your code on a PCD file.The options are:
I -n Estimate surface normalsI -k Detect keypointsI -l Compute local descriptors
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Questions?
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
example
The slides after this point are random ones we didn’t end upusing...
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
PFH (5/5)Complexity Analysis
Complexity is high: O(k2). Optimizations to the rescue!
Unordered OrderedPoint Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Fast PFH (1/4)Basic Concepts
Re-formulate: FPFH(p) = SPF (p) + 1k∑k
i=11ωk· SPF (pk )
Point Feature Histograms (PFH) Fast Point Feature Histograms (FPFH)
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Fast PFH (2/4)Theoretical formulation
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Fast PFH (3/4)Noise Analysis
Synthetically noiseless Synthetically noisified
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Fast PFH (4/4)Noise Analysis
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Features (1/4)pcl::NormalEstimation<T> p;
I p.setInputCloud (data);
p.SetRadiusSearch (0.01);
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Features (2/4)pcl::NormalEstimation<T> p;
I p.setInputCloud (data);
p.SetRadiusSearch (0.01);
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Features (3/4)pcl::BoundaryEstimation<T,N> p;
I p.setInputCloud (data);p.setInputNormals (normals);
p.SetRadiusSearch (0.01);
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Features (4/4)pcl::PrincipalCurvaturesEstimation<T,N> p;
I p.setInputCloud (data);p.setInputNormals (normals);
p.SetRadiusSearch (0.01);
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Feature PersistenceWhat scale to choose ?
bad scale (too small) good scale
Selecting the right scale (k -neighborhood) is problematic:
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local Features (4-5/5)Consistent Normal Orientation
Before
I Extended Gaussian ImageI Orientation consistent for:
1. registration2. feature estimation3. surface representation
I normals on theGaussian sphere
I should be in thesame half-space
Point Cloud Library (PCL)
Introduction Feature Estimation Keypoint Detection Combining Keypoints and Features
Local Features (4-5/5)Consistent Normal Orientation
Before After
(vSolet ′slookatsomecode : iewpoint − pi) · npi ≥ 0
or:
propagate consistencythrough an EMST
Point Cloud Library (PCL)