Mikio Sakemoto Intel K.K....Mikio Sakemoto Intel K.K. 2 Agenda ... Robotics is Compute-Intensive!...

Autonomous Mobile Robotics with Intel® RealSense™ Camera and ROS*

Mikio SakemotoIntel K.K.

2

Agenda• Robotics Overview

• Intel® Hardware for Robotics

• Robot Designs

• ROS* Concepts and Navigation

• Conclusion

© 2017 Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. For more complete information about compiler optimizations, see our Optimization Notice.

Robots come in many forms…

https://software.intel.com/en-us/articles/optimization-notice#opt-en


Robots that interact with people need specialized skills:• Speech recognition

• Face recognition

• Dialogue management

• Context awareness

• Emotional state recognition

• Social relationship understanding

Social Robotics



Mobile robots need to:

• Sense a potentially dynamic environment

• Determine current location

• Navigate from one location to another

• Also need to interact with humans as needed (avoid, follow, respond to commands, etc.)

Mobile Robotics



Many Intel-Based Robot Designs!



Savioke Relay with Intel® RealSense™



8

CES 2016: Ninebot* and Intel Segway* Robot

http://robot.segway.com/


http://robot.segway.com/

Typical Robot Software Stack

ServosWheel Motors

RGB Camera

Depth Camera

Motion Module

Touch Mic Array

Discrete SensorsIntel® RealSense™ Camera Drivers Motor Drivers

Senso

r Fusio

n

Point Cloud

2DMap

Body Tracking

Face Tracking

Object Recognition

RGBD+IMU / SLAM

Speech RecognitionBehaviorControl

Reflex

Goals

Interact

SpeechOutput

DialogControl

User/ World Knowledge

Gesture / Action

WorldView

RobotState

CommandInputs

Behavior Arbitrator

Path Planning

Movement

External Device Control

Voice CommandExternal

Controller

Gesture InputEnvironment

StimulusBehavior

BehaviorBehaviors

Applications

Acoustics

Op

tion

al Inp

uts

Cloudoffload &control

IRUltra-Sonic

Sense Plan Act

Sensor Hardware and Drivers Actuators

Robotics is Compute-Intensive!

ServosWheel Motors

RGB Camera

Depth Camera

Motion Module

Touch Mic Array

Discrete SensorsIntel® RealSense™ Camera Drivers Motor Drivers

Senso

r Fusio

n

Point Cloud

2DMap

Body Tracking

Face Tracking

Object Recognition

RGBD+IMU / SLAM

Speech RecognitionBehaviorControl

Reflex

Goals

Interact

SpeechOutput

DialogControl

User/ World Knowledge

Gesture / Action

WorldView

RobotState

CommandInputs

Behavior Arbitrator

Path Planning

Movement

External Device Control

Voice CommandExternal

Controller

Gesture InputEnvironment

StimulusBehavior

BehaviorBehaviors

Applications

Acoustics

Op

tion

al Inp

uts

Cloudoffload &control

IRUltra-Sonic

Sense Plan Act

Sensor Hardware and Drivers Actuators

Co

mp

ute!

Intel Focus


Key Robotics Sensing RequirementsUnderstanding Environment Interaction with ObjectsInteraction with People

Map CreationLocalization (SLAM)Collision AvoidanceNavigationObject and Person Location

Person DetectionPerson TrackingPerson RecognitionPosture AnalysisBody Gestures

Object Segmentation Object Recognition3D ReconstructionObject Localization



6DoF Tracking Re-localizationMapping • Track with no prior information

about the environment• Robust to moving or displaced

objects in scene• Robust to loss and degradation of

sensor inputs

• Integrates geometry and depth information

• re-localizes to a pre-defined coordinate system

• 2D Occupancy Map during robot exploration

• Dynamically updated as the environment changes

• Registered to a SLAM Map, share across systems

Navigation Capabilities - SLAM



SLAM with 2D Mapping

SLAM Created Navigation Map



SLAM with 3D Reconstruction


15



• Robot Designs


• Conclusion


Generic Robot Hardware Architecture

Master Computer

Linux*

Slave MCU

RTOS

Vision

Audio

Actuators

Sensors

Network

Localization



Intel® RealSense™ D400 Series

https://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html

• Intel® RealSense™ Depth Camera D400 Series are ready-to-use right out of the box and can be easily added into your existing prototypes via USB. Bring innovative visual solutions to your projects with enhanced, accurate computer vision by adding high image-resolution and high frame-rate capabilities. Both Intel® RealSense™ Depth Cameras include long-range vision and work in both indoor and outdoor settings.

• librealsense and realsense_ros_camera for D400 Series are available at github.



• Powerful computer in a small package

• Intel® Core™ i5/i7 processors

• Large and configurable memory, drive size, etc.

• Wi-Fi* and Bluetooth*, 4K video, USB3.0 ports, Gen acceleration…

• Broad OS support, including Linux*

Intel® NUC

Maximum performance for robotics development



19

OEM Boards for robotics

Rapid prototype-to-production for robotics development

http://www.up-board.org/upsquared/


20



• Robot Designs


• Conclusion


SAWR Basic Mobile RobotComputer: UP* Board or OEM board

Camera: Intel® RealSense™ Camera

Actuation: Two Dynamixel MX-12W smart

servos with magnetic encoders

Software: Xubuntu and ROS

Frame: Laser-cut acrylic or POM, Polulo sphere

casters, o-ring tires and belt transmission

Other: DF Robot power regulator

Extras: Jabra* 510+ USB speakerphone

GO TO: https://github.com/01org/sawr



01.org/sawr



Can Build sawr with Either UP Board or OEM board



Development Process

• Software stack based on modification and simplification of the OSRF*

Turtlebot* stack

• Custom motor driver written based directly on Dynamixel* Linux* SDK

• Frame parametrically modelled using OpenSCAD*

and then converted to lasercut files using Inkscape*



Laser Cut Frame

One 450 x 300 x 3mm sheet and one 450 x 300 x 2mm sheet



DIY… FOSS

• Detailed instructions, assembly, and setup plans available online…

• Open-source HW and SW



Robot Architecture: with UP BoardMaster

Aaeon* UP* Board

Slave

Robotis* Dynamixel*

MX-12W Smart Motors

Intel® RealSense™ camera

Audio I/O (Jabra* 510+) [optional]

USB3.0

USB2.0

UART (via USB2AX)

Full-bridge motor driver (PWM)

Magnetic quadrature encoder

Torque sensing (motor current)

Gb Eth

HDMI

Wi-Fi*/Bluetooth*USB2.0



Using OEM board for Robotics

• Typically want active cooling to get higher performance• Can maintain CPU highest speed indefinitely• Power usually not a limiting factor in

robotics (as motors are usually the primary power load)

• Want to run Ubuntu and ROS• You can run ROS on Ostro, but…

• Compared to UP Board• Has higher performance• Built-in WiFi and BT



And Now: Turtlebot 3!



Turtlebot 3*

• Joint effort between Intel, Robotis*, and OSRF*

• Modular, can support many custom configurations



Multiple Camera Support

• Possible to run up to 3 Intel® RealSense™ cameras• Although with more than

two, frame rate reduction is necessary

• Intel® RealSense™ZR300 drivers for Linux*

only


32



• Robot Designs


• Conclusion


ROS*: Robot Operating System

• Supports distributed system based on a graph of communicating nodes

• Large community with many existing nodes for sensors, planning, navigation, etc.

• Various visualization tools and simulators are also provided



Autonomous Navigation ROS* Node Graph



Basic ROS* Concepts: Messages and Topics• Publish/subscribe data on uniquely named topics

• Topics can have multiple publishers and subscribers

• Messages are typed, can carry multiple elements

Node Node

Topic

Publish Subscribe



Basic ROS* Concepts: Messages and Topics

NodeNode

Topic

Node

Node

Node



Basic ROS* Concepts: Service Calls• Synchronous remote procedure call

• Caller blocks until response received

• Of limited use: synchronization, get/set

Node Node

Service Call: Request

Service Call: Response



ROS* Concepts: Actions• Asynchronous goal-directed behavior control based on standard set of

topics

• Set goals, initiate actions, monitor progress, cancel if necessary, be informed of success or failure

Node Node

Action: Goal

Action: Progress

Action: Progress

Action: Success



Basic ROS* Concepts: Parameters• Parameter server provides typed constant data at startup

• Parameters can be specified in various ways: command line, parameter files, launch file parameters, etc.

• Newer version of ROS provide dynamic parameter update

Node Node

Parameters

39



Other ROS ConceptsPackages:

• Collections of files used to implement or specify a service or node in ROS*, built together using catkin (typically)

URDF (Universal Robot Description):

• XML files describing joints and transformations between joints in a 3D model of robot

Launch Files:

• XML files describing a set of nodes and parameters for a ROS graph

YAML (Yet Another Markup Language):

• Used for parameter specification on the command line and in files



ROS* ToolsRviz:• Visualize various forms of dynamic 3D data in context: transforms, maps, point

clouds, imagesGazebo:• Robot simulation, including collisions, inertia, perceptual errors, etc.

Rqt:• Visualize graphs of nodes and topicsCommand-line tools:

• Listen to and publish on topics, make service calls, initiate actions• Filter and monitor error messagesCatkin:• Build system and package management



Rviz: Robot Visualization



ROS* common modules for autonomous movetf(tf2):• Coordinate transform library.slam_gmapping:• ROS wrapper for OpenSlam's Gmapping.move_base:• planning route, maintain cost maps, and navigationamcl:• probabilistic localization systemRobot_state_publisher:• publishes the 3D poses of the robot links



tf frame



Hints for custom robot using sawr stack…

• sawr_master: Master package, launch scripts.• Modify if you change modules.

• sawr_description: Run-time physical description (URDF files).• Modify urdf and xacro files according to your robot dimension (check with tf tree/frame)

• sawr_base: Motor controller and hardware interfacing.• Prepare your own motor controller and odometry libraries

• sawr_scan: Camera configuration.• You can use as-is in most cases

• sawr_mapping: SLAM configuration.• You can begin from as-is if you use same hardware configuration with sawr.

• sawr_navigation: Move-base configuration.• Modify and tune parameters of global/local costmap, move_base.


https://github.com/01org/sawr/blob/master/sawr_master/README.md

https://github.com/01org/sawr/blob/master/sawr_description/README.md

https://github.com/01org/sawr/blob/master/sawr_base/README.md

https://github.com/01org/sawr/blob/master/sawr_scan/README.md

https://github.com/01org/sawr/blob/master/sawr_mapping/README.md

https://github.com/01org/sawr/blob/master/sawr_navigation/README.md


Hints for custom robot using sawr stack…RealSense (sawr_scan)

State publisher (sawr_description)

SLAM (sawr_mapping)

Navigation (sawr_navigation)

Motor(sawr_base)

Transform



Rviz demo move of SAWR based robot


48



• Robot Designs


• Conclusion


Conclusions• Autonomous mobile robotics is an emerging area

• Technology for mobile robotics is now relatively mature• Expect to see autonomous vehicles on the road soon!

• Robotics involves all aspects of computer science and engineering• AI, computer vision, machine learning, speech understanding, IoT,

networking, real-time control…

• Intel provides many key components enabling sophisticated robotics

• ROS* is a key framework for robot SW development



Additional Sources of Information

• OSRF: http://www.osrfoundation.org/

• ROS*: http://ros.org and http://wiki.ros.org

• SAWR: http://01.org/sawr or http://github.com/01org/sawr

• Turtlebot 3: http://turtlebot3.robotis.com/en/latest/hardware.html

• Recommended books: 1. Programming Robots with ROS: A Practical Introduction to the Robot

Operating System, by Morgan Quigley and Brian Gerkey. O’Reilly, 2015.2. Learning ROS for Robotics Programming, 2nd Edition, by Enrique Fernandez

and Luis Sanchez Crespo, PACKT, 20153. Mastering ROS for Robotics Programming, by Lentin Joseph, PACKT, 2015


http://www.osrfoundation.org/

http://wiki.ros.org/

http://01.org/sawr

http://github.com/01org/sawr

http://turtlebot3.robotis.com/en/latest/hardware.html


Backup



インテル® Parallel Studio XEベータ2018の

無料の30日間試用版を今日ダウンロードします>固有のタグ付けを持つ貴社のカスタムURLを以下のサンプルリンクに挿入してください

https://intelcustomer.az1.qualtrics.com/SE/?SID=SV_09AEJgAYdKezL6d&Q_JFE=0

忘れずに以下のことを行ってください...

貴社の商品とURLの宣伝に、どうぞこのスライドをテンプレートとしてご使用ください例えば、上の例はインテル® Parallel Studio XEベータ2018の販促用のものとなっています

卓越した性能を持つコード

このプレゼンテーション後に評価調査をＥメールで送信いたしますので、

受信トレイ内をご確認ください。

追記アンケートにご記入いただいた方全員に、研修完了を示す個人の証明書をお送りいたします！

商品ボックスの写真をここに挿入してください。



Legal Notices and Disclaimers

• Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.

• Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance.

• Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit http://www.intel.com/performance.

• Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.

• This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.

• No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

• Statements in this document that refer to Intel’s plans and expectations for the quarter, the year, and the future, are forward-looking statements that involve a number of risks and uncertainties. A detailed discussion of the factors that could affect Intel’s results and plans is included in Intel’s SEC filings, including the annual report on Form 10-K.

• All products, computer systems, dates and figures specified are preliminary based on current expectations, and are subject to change without notice. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

• Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.

• © 2016 Intel Corporation. Intel, the Intel logo, Intel Core, Intel. Experience What’s Inside, Intel. Experience What’s Inside Logo, RealSense are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

• *Other names and brands may be claimed as the property of others.


http://www.intel.com/performance

http://www.intel.com/performance

Mikio Sakemoto Intel K.K....Mikio Sakemoto Intel K.K. 2 Agenda ... Robotics is Compute-Intensive!...

Documents

Transcript of Mikio Sakemoto Intel K.K....Mikio Sakemoto Intel K.K. 2 Agenda ... Robotics is Compute-Intensive!...