Email: [email protected], [email protected], [email protected]
Presentazione standard di PowerPoint - PCL · . Live Demo! Hands-on session ....
45
Transcript of Presentazione standard di PowerPoint - PCL · . Live Demo! Hands-on session ....
People Detection
RGB-D data
People Detection in RGB-D Data
Dense scanning of RGB and depth images [1] (GPU processing required for real time)
[1] L. Spinello and K. O. Arras. People detection in rgb-d data. In IROS 2011. [2] D. Mitzel and B. Leibe. Real-time multi-person tracking with detector assisted structure propagation. In ICCV Workshops, 2011.
ROI tracking and dense scanning of the ROIs [2]
Related Work
Objectives
detecting people walking on a ground plane
from a mobile robot
in real time with standard CPU
Our approach [3]
[3] M. Munaro, F. Basso, and E. Menegatti. Tracking People within Groups with RGB-D Data. In IROS 2012.
Ground plane estimation and removal
Detection on point cloud
Euclidean clustering of remaining points
Ground plane estimation and removal
Detection on point cloud
People vertically splitted into more clusters
Main issues
People vertically splitted into more clusters
Main issues
People vertically splitted into more clusters
More people merged into the same cluster
Main issues
People vertically splitted into more clusters
More people merged into the same cluster
Main issues
People vertically splitted into more clusters
More people merged into the same cluster
Main issues
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
3. Subdivision of big clusters by means of head detection (peaks in height from the ground plane)
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
3. Subdivision of big clusters by means of head detection (peaks in height from the ground plane)
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
3. Subdivision of big clusters by means of head detection (peaks in height from the ground plane)
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
3. Subdivision of big clusters by means of head detection (peaks in height from the ground plane)
Sub-clustering procedure
Sub-clustering procedure
1. Candidates pruning based on height from the ground plane
2. Merging of clusters close in ground plane coordinates
3. Subdivision of big clusters by means of head detection (peaks in height from the ground plane)
Candidate clusters are extended until the ground plane for achieving robustness to lower limbs occlusion
HOG+SVM evaluation
Candidate clusters are extended until the ground plane for achieving robustness to lower limbs occlusion
HOG+SVM evaluation
Candidate clusters are extended until the ground plane for achieving robustness to lower limbs occlusion
HOG+SVM evaluation
[4] N. Dalal and B. Triggs. Histogram of Oriented Gradients for Human Detection, CVPR 2005.
HOG+SVM evaluation
Candidate clusters are extended until the ground plane for achieving robustness to lower limbs occlusion HOG detector [4] applied to image patches that are projection of 3D clusters onto the RGB image
All clusters
With merging and height constraint
With dimension constraints
With HOG constraint
HOG confidence
GroundBasedPeopleDetectionApp<PointT>
PersonClassifier<pcl::RGB> HeadBasedSubclustering<PointT>
HeightMap2D<PointT> HOG
PersonCluster<PointT>
pcl::people
#include <pcl/people/ground_based_people_detection_app.h>
…
// Create classifier for people detection:
pcl::people::PersonClassifier<pcl::RGB> person_classifier; person_classifier.loadSVMFromFile(svm_filename); // load trained SVM
pcl::people
#include <pcl/people/ground_based_people_detection_app.h>
…
// Create classifier for people detection:
pcl::people::PersonClassifier<pcl::RGB> person_classifier; person_classifier.loadSVMFromFile(svm_filename); // load trained SVM
// People detection app:
pcl::people::GroundBasedPeopleDetectionApp<PointT> people_detector; // people detection object
std::vector<pcl::people::PersonCluster<PointT> > clusters; // vector containing persons clusters
people_detector.setIntrinsics(rgb_intrinsics_matrix); // set RGB camera intrinsic parameters
people_detector.setClassifier(person_classifier); // set person classifier
people_detector.setInputCloud(cloud);
people_detector.setGround(ground_coeffs); // set floor coefficients
people_detector.compute(clusters); // perform people detection
pcl::people
// Optional settings:
people_detector.setVoxelSize(voxel_size); // set the voxel size (0.06)
people_detector.setPersonClusterLimits(min_height, max_height, min_width, max_width); // set dimension limits for a person cluster (1.3, 2.3, 0.1, 8.0)
people_detector.setMinimumDistanceBetweenHeads(min_dist); // set minimum distance between persons’ heads (0.3)
people_detector.setSamplingFactor(sampling_factor); // downsampling factor for the point cloud (1)
people_detector.setSensorPortraitOrientation(true); // set portrait/landscape orientation (false)
pcl::people
// Display pointcloud: pcl::visualization::PointCloudColorHandlerRGBField<PointT> rgb(cloud); viewer.addPointCloud<PointT> (cloud, rgb, "input_cloud"); // Add point picking callback to viewer: struct callback_args cb_args; PointCloudT::Ptr clicked_points_3d (new PointCloudT); cb_args.clicked_points_3d = clicked_points_3d; cb_args.viewerPtr = pcl::visualization::PCLVisualizer::Ptr(&viewer); viewer.registerPointPickingCallback (pp_callback, (void*)&cb_args); // Spin until 'Q' is pressed (to allow ground manual initialization): viewer.spin();
Manual ground initialization
Automatic ground initialization
OrganizedMultiPlaneSegmentation
Input point cloud
Check on horizontality
The lowest plane is selected
Automatic ground initialization
// Ground plane estimation: pcl::people::GroundplaneEstimation<PointT> ground_estimator (ground_estimation_mode); Eigen::VectorXf ground_coeffs; ground_estimator.setInputCloud (cloud); ground_coeffs = ground_estimator.compute ();
Automatic ground estimation
ground_estimation_mode:
0: manual estimation
1: semi-automatic
2: automatic with visualization
3: automatic without visualization
// CMakeLists.txt cmake_minimum_required(VERSION 2.8 FATAL_ERROR) project(pcl_ground_based_rgbd_people_detection) find_package(PCL 1.7 REQUIRED) include_directories(${PCL_INCLUDE_DIRS}) link_directories(${PCL_LIBRARY_DIRS}) add_definitions(${PCL_DEFINITIONS}) add_executable (pcl_ground_based_rgbd_people_detection src/main_ground_based_people_detection.cpp) target_link_libraries (pcl_ground_based_rgbd_people_detection ${PCL_LIBRARIES})
Test app
// CMakeLists.txt cmake_minimum_required(VERSION 2.8 FATAL_ERROR) project(pcl_ground_based_rgbd_people_detection) find_package(PCL 1.7 REQUIRED) include_directories(${PCL_INCLUDE_DIRS}) link_directories(${PCL_LIBRARY_DIRS}) add_definitions(${PCL_DEFINITIONS}) add_executable (pcl_ground_based_rgbd_people_detection src/main_ground_based_people_detection.cpp) target_link_libraries (pcl_ground_based_rgbd_people_detection ${PCL_LIBRARIES}) // Execution: $ ./pcl_ground_based_rgbd_people_detector
Test app
Results
Results
Input XYZRGB point cloud
Ground coefficients
Output People clusters (centroid, points indices, …)
Framerate (on Intel i7-3630QM @ 2.4GHz) About 15 fps (VGA resolution, data grabbing and visualization not
included)
About 30 fps (with QQVGA resolution)
Input/Output
People detection in unconventional poses:
Crouching
Hands up
Sitting
Dancing/running
Lying on sofa/floor
Student: Lingzhu Xiang, Brown Univ.
Google Summer of Code 2014
http://www.pointclouds.org/blog/gsoc14
Live Demo!
Hands-on session
