IROS 2012 Handling ROS tutorial Vilamoura, October 7th 2012 http ...
-
Upload
nguyendien -
Category
Documents
-
view
223 -
download
1
Transcript of IROS 2012 Handling ROS tutorial Vilamoura, October 7th 2012 http ...
S i l v i a R o d r í g u e z - J i m é n e z
C a r l o s I I I U n i ve r s i t y o f M a d r i d
V i s u a l P e r c e p t i o n S ys t e m w i t h i n H AN D L E E U p r o j e c t
u s i n g R O S
IROS 2012 Handling ROS tutorial
Vilamoura, October 7th 2012 http://mrl.isr.uc.pt/events/iros2012tutorial/
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 2
Robots need visual 3D perception
Picture from http://thecorpora.com
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 3
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 4
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 5
Depth measurement techniques
Picture from the Artist Audrey Penven
Depth Measurement
Techniques
Microwaves Light
Waves Ultrasonic
Waves
Triangulation Time-
of-Flight
Structured
light
Laser
Scan
Kinect
PrimeSense Asus Xtion
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 6
Principles of Kinect
640x480 px @ 30 fps
Operation range:
0.8m~3.5m
Depth resolution [1]:
IR Laser projector
λ=830nm
RGB
camera
IR camera
Picture from iFixit
Theoretical random error
Depth resolution
Distance to plane [cm]
Depth image RGB image
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 7
How Kinect works?
Libraries: OpenNI, libfreenect y Kinect SDK
Picture from PrimeSense
Projected Light Pattern [2] 3D Reconstruction
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 8
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 9
OpenNI in ROS
Picture from [3]
http://www.ros.org/wiki/
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 10
openni_kinect
ROS/Kinect integration
openni_tracker openni_camera
openni_launch image_pipeline
vision_opencv
perception_pcl
3D
2D
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 11
ROS image and OpenCV image
OpenNI:
RGB format
OpenCV:
BGR format
http://www.ros.org/wiki/cv_bridge/Tutorials/UsingCvBridgeToCon
vertBetweenROSImagesAndOpenCVImages
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 12
ROS/Kinect integration
Start the Kinect driver:
roslaunch openni_launch openni.launch
Reconfigure camera parameters:
rosrun dynamic_reconfigure
reconfigure_gui
Visualize the images:
rosrun rviz rviz
If you have problems with starting rviz, remove files in ~/.rviz/.
Add a PointCloud2 View: XYZRGB point cloud
For topic camera/depth_registered/points
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 13
ROS/Kinect integration
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 15
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 16
Calibrating a Kinect to your robot
TF
Hands-on #2
Session
Checkerboard detector
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 17
ROS Examples
[4]
[5]
[6]
[7]
[8]
[9]
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 18
System for robotic visual perception
Segmentation, recognition, pose estimation, object
tracking…
ROS stacks:
http://www.ros.org/wiki/iap: tracking and segmenting visual
features
http://www.ros.org/wiki/perception_blort: object recognition and
pose estimation based on detection and tracking nodes.
http://www.ros.org/wiki/tabletop_object_perception: object
segmentation, object recognition and collision environment
PCL and OpenCV.
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 19
Table detection using RANSAC
Segmentation
Picture from PCL
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 20
Object recognition and pose estimation
Picture from PCL
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 21
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 22
Interfacing with a SQL database
Installed:
A PostgreSQL server.
A database: restored from the provided backup file.
Iros_st2_database package: functions for interfacing with a
specific SQL database.
Iros_st2_database_msgs package: ROS API.
Updated model_root:
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 23
SQL database: schema
Starting point: household_objects_database
www.ros.org/wiki/household%20objects
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 24
SQL database: usage
Launch the local server:
roslaunch iros_st2_database iros_st2_database_server.launch
Provided ROS services:
iros_st2_hdb/get_list_by_acquisition_method: list of models
available in the database by given an acquisition method
iros_st2_hdb/delete_model: deletes a model
iros_st2_hdb/download_model: gets a 3D mesh
iros_st2_hdb/insert_model: stores a model in the database
iros_st2_hdb/update_model: updates a stored model
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 25
SQL database: ROS Services
Get list by acquisition method:
rosservice call iros_st2_hdb/get_list_by_acquisition_method
<name>
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 26
SQL database: ROS Services
Download a mesh from the database:
rosservice call iros_st2_hdb/download_model
<scaled_model_id>
Insert a model in the database, copying the files to a location
specified by the model_root.
rosservice call iros_st2_hdb/insert_model <model_name>
<acquisition_method> <maker> <description> <barcode>
<geometry_filename> <color_image_filename>
E.g.: rosservice call iros_st2_hdb/insert_model camera iros_st2 “” “” “”
/iros_st2_database/database/examples/camera.model/mesh.ply
/iros_st2_database/database/examples/camera.model/view0000/raw/color.
png
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 27
SQL database: ROS Services
Delete a model from database:
rosservice call iros_st2_hdb/delete_model <scaled_model_id>
Update a model from database:
rosservice call iros_st2_hdb/update_model <original_model_id>
<model_name> <acquisition_method> <maker> <description>
<barcode> <geometry_filename> <color_image_filename>
E.g.: rosservice call iros_st2_hdb/update_model 9623 camera iros_st2
unknown “” “” “” “”
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 28
Outline
Depth measurement techniques: Kinect
ROS/Kinect integration
System for robotic visual perception
Interfacing with an object SQL database
ROS vision nodes within HANDLE EU project
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 29
HANDLE: Kinect/robot calibration
Kinect
Camera
Shadow
Arm
Shadow
Hand
ATI
Force/Torque
Sensors
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 30
HANDLE: reconstruction using a single view
Color and Depth Image
Table-top Object Detector
Voxel filling by extrusion
Consistency Check
3D Object Model
Color-based Model Refinement
Poisson Reconstruction
3D Scene Points
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 31
HANDLE: reconstruction using a single view
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 32
HANDLE: Object recognition and pose
Viewpoint Feature Histogram (VFH) descriptor used for matching and recognition of point clouds [10]
Recognition and 6D pose estimation for objects lying on a table:
Database
ICP
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 33
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 34
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 35
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 36
HANDLE: Object recognition and pose
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 37
References
[1] K. Khoshelham and S. O. Elberink. Accuracy and Resolution of Kinect
Depth Data for Indoor Mapping Applications. Sensors, 12(2):1437--1454, 2012.
[2] www.ros.org/wiki/kinect_calibration/technical
[3] http://openni.org/Documentation/ProgrammerGuide.html
[4] www.ros.org/wiki/camera_pose_toolkits/Tutorials/Add_Kinect_to_PR2
[5]
www.ros.org/wiki/openni/Contests/ROS%203D/Automatic%20Calibration%20of
%20Extrinsic%20Parameters
[6]
www.ros.org/wiki/turtlebot_kinect_arm_calibration/Tutorials/CalibratingKinectTo
TurtleBotArm
[7] www.ros.org/wiki/pr2_calibration/Tutorials/NullChainKinectURDFCalibration
[8] www.ros.org/wiki/pr2_calibration/Tutorials/KinectURDFCalibration
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 38
References
[9] www.ros.org/wiki/camera_pose_toolkits/Tutorials/Calibrate_camera_pose_to
_map
[10] R. Rusu, G. Bradski, R. Thibaux and J. Hsu. Fast 3d recognition
and pose using the viewpoint feature histogram. In Proc. of IROS 2010, pages
2155-2162, 2010.
Tutorial ST2 IROS 2012 | Hands-on #1 | Silvia Rodríguez-Jiménez 39
Thank you!
Questions/comments?