Sense. Reason. Act.on-demand.gputechconf.com/gtc-eu/2018/pdf/e8208...Dr. Michael Suppa I Co-Founder...

Dr. Michael Suppa I Co-Founder and CEO

Sense. Reason. Act.Real-time Perception, Navigation, and Manipulation for Robotic Systems

Roboception GmbH - confidential - www.roboception.de

about rc_visard rc_reason applications thank youinnovation


Founded 03/2015 by three former employees of the Institute of Robotics and Mechatronics/ German Aerospace Center as a

KUKA Deutschland GmbHShareholder since 08/2015

Based in MUNICH (Pasing)

18 employees (08/2018)

Company ProfileSENSE. REASON. ACT

DLR SPIN-OFF



Mission

Going from pixel to action using perception.

Sense. Reason. Act.



3D Perception – Why?

“3D vision is a DISRUPTIVE TECHNOLOGY that enables robots to operate in a partially-structured environment”.

IFR Annual Report on Service Robotics

“Perception was the DOMINATING FACTOR separating the winners from the rest of the field in the amazon picking challenge.”

Henrik Christensen

Sources: 1. http://www.ifr.org/robots-create-jobs/2. http://www.therobotreport.com/news/amazon-challenges-robotics-hot-topic-perception3. http://www.robotics.org/content-detail.cfm/Industrial-Robotics-Industry-Insights/Intelligent-Robots-A-Feast-for-the-Senses/content_id/5530

www.roboception.de

"If we were only able to provide the VISUAL CAPABILITIES of a 2-year old child, robots would quickly get a lot better.“

Rodney Brooks



TOMORROW’S ROBOTS operate in partially structured environments and…

Market RequirementsLOW-COST AND COLLABORATIVE ROBOTS ARE THE FUTURE

…are LOW-COST systems

… safely COLLABORATEwith the human

… enable transparent, SECURE DATA EXCHANGE

…. rely on 3D VISION and tightly coupled perception and manipulation

… are able to support DECISION-MAKING

… are INTUITIVE and integratable, with easy-to-use functionalitiesPLUG-AND-PLAY

… handle and PROCESS sensor data in (NEAR-) REAL TIME



Hardware/ basic software enhanced bymodular software components (> scalable)

3D information in real-time, on-board processing

Software products for real-time perceptionand manipulation of robotic systems

Navigation and perception solutions for innovative applications

Intuitive: No expert knowledge required

Business ModelVERSATILE SENSOR AND INTUITIVE ROBOTICS SOFTWARE FOR REAL-TIME 3D

rc_visard Sensor Family

rc_visard Standard Software Package

rc_reason Software Suite

SLAM TagDetect ItemPick […]

SEN

SE. R

EASO

N. A

CT.



PIPELINE, INCLUDING MANIPULATION AND GRASPING

Motion planning:Collision‐free path

Grasp/ manipulation execution:Arm/ gripper control

Supervisory system:Error detection

Planning of corrective actions

Grasp planning:Known objects: grasp database

Familiar objects: grasp adaptationUnknown objects: online

grasp planning for the approx. geometry

Object model database: CAD model (mesh/ object coordinate frame)

Semantic annotations (action/ restriction areas)Mass/ inertial properties

Grasp database (gripper dependent)

Scene registration:Pointcloud

B&W/ RGB images

Scene interpretation:Localization wrt. environment

Plane detection (tabletop scenarios)Clustering

Object recognition/ approximation3D/ 6D object pose estimation

(application dependent)

VISUAL PERCEPTION

MANIPULATION EXECUTION

Task information: Goal pose/ intended use

IK/ Reachability map of the manipulator

Task information: Gripper specification

Required hand / power tools

Grasp sorting:Reachability

Feasibility for start/ goal configuration

Grasp quality

CONTEXTUAL INFORMATION

GRASP / MOTION PLANNING

Feedback:Visual (object/ hand tracking)

Tactile (gripper)

CLOSURE OF THE ACTION/PERCEPTION LOOP

Closing the Perception-Action-LoopPIPELINE INCLUDING MANIPULATION AND GRASPING



Sensor in four variants (baseline 65 mm / 160 mm, monochrome/ color capacity) delivering reliable 3D data for the perception-action-loop

• Stereo, depth images and ego-motion

• On-board processing capabilities

• Intuitive plug-and-play interfaces

rc_visard Sensor Family3D INFORMATION IN REAL-TIME



From Prototype to Industrial StandardPRODUCT DEVELOPMENT TIMELINE

8/2015 12/2015 6/2016 12/2016

Intel-based prototype

ARM-based prototype

Product definition

Final design

Product launch

Collaboration with research partners in various applications

Proof of concept

1st financing round with strategic partner KUKA

Production partner

Kick-Off 06/2017

Product development

SOP



Product ConceptPERCEPTION AND NAVIGATION SOLUTIONS

Four versions: rc_visard 65 m I 65 c I 160 m I 160 c• Stereo and depth images as well as ego-motion• On-board processing, no external hardware• Intuitive plug-and-play interfaces• Precise determination of position and orientation • Low latencies• Works in natural and artificial light• Reliable ego-motion (even in case of vibrations)• Multiple sensors operate in same work space• Robust fusion of stereo an inertial data

• object recognition• indoor navigation• grasp planning• world modelling


Image Resolution 1280 x 960 Pixel, 1.2 Mpixel

Baseline rc_visard 65: 65 mm rc_visard 160: 160 mm

Field of View 48° x 61°

Depth Range rc_visard 65: 0.2–1 m rc_visard 160: 0.5-3 m

Depth Resolution rc_visard 65: 0.5-10 mm rc_visard 160: 1.5-50 mm

Depth Image Resolution 640x480 (H), 320x240 (M), 214x160 (L)

Frame Rate 3 Hz (H), 15 Hz (M), 25 Hz (L)

Ego-Motion 200 Hz, low latency

Computing Unit Nvidia Tegra K1

Power Supply 18-30 V

Protection Class IP54

Operating Temperature 0-50°C, passive cooling


rc_visard 65 & rc_visard 160PRODUCT SPECIFICATION



rc_v

isar

dST

EREO

DAT

A PR

OCE

SSIN

G R

ESU

LTS



rc_v

isar

dEg

o-M

otio

nPO

INT

CLO

UD

GEN

ERAT

ION



rc_reason Software SuiteON-BOARD ENHANCEMENT OF BASIC SOFTWARE

SLAM: Precise mapping and pose estimation in 3D for a drift-free navigation

TagDetect: Identification and pose estimation of QR codes and AprilTags

ItemPick: Computation of grasp poses for suction grippers in pick-and-place applications

& MORE TO COME



rc_reason SLAM ModuleSIMULTANEOUS LOCALIZATION AND MAPPING

rc_visard captures its environment in a map and simultaneously estimates its current location for optimized navigation.

• Precise mapping and pose estimation in 3D• Enables a recognition of previously visited

places• Drift-free: Online correction of trajectory• Out-of-the-box: Easy installation and

operation



rc_r

easo

nSL

AM/ R

OBO

T N

AVIG

ATIO

N



Rapid detection of QR codes and AprilTags for a reliable detection of objects, e.g. in pick-and-place applications.

• Identification/localization of labelled objects• Reliable localization of a sensor in 3D space• Time-efficient: Detection time < 1 sec• On-board data processing: No external

hardware required• Out-of-the-box: Easy installation/operation

rc_reason TagDetect Module



rc_r

easo

nTa

gDet

ect



Computes surface grasp poses for a suction device on a given object.• Identification of load-carrier for bin picking• Reliable detection of flat surface(s) on items• Works with static and robot-mounted sensors• On-board data processing: No external

hardware required• Out-of-the-box: Easy installation and

operation

rc_reason ItemPickFOR ROBOTIC PICK-AND-PLACE APPLICATIONS USING SUCTION GRIPPERS



Object RecognitionGPU/CPU-BASED OFF-BOARD COMPUTING

Object recognition• Object identification & manipulation through

advanced algorithms• Distance is determined through depth data

Object database • Stores object data (geometry, features,

associated handles)• Database can be updated/extended by the

user at any time



Computing RessourceTK1 VS XAVIER FOR STEREO DATA PROCESSING

Model Settings Unit SGM 640x480x256 SGM 1280x960x384 Speedup

Time [ms] FPS[Hz] Time [ms] FPS[Hz]

TK1 CPU:1836 MHzGPU: 804 MHz<15 W

CPUGPU

1559280

0.63.6

97591571

0.10.6

1.01.0

Xavier Mode: 10W CPUGPU

2742164

0.46.1

14096888

0.11.1

0.61.7

Xavier Mode: 15W CPUGPU

136274

0.713.5

5850506

0.22.0

1.43.4

Xavier Mode: 30W_ALL CPUGPU

68054

1.518.5

2804382

0.42.6

2.94.6

Xavier Mode: NMAX CPUGPU

47137

2.127.0

1818275

0.63.6

4.36.6



Model-free robotic bin-pickingOFFBOARD COMPUTING OF GRASPS

Stereo‐Matching ItemPick

rc_visard Jetson Xavier

left and rightimage

imageacquisition andrectification

depth imagecomputation

(SGM)

bin detection, surface

clustering, grasp

computation

From pixels to grasps in ~ 1 second!



Configuration, Calibration and OperationINTUITIVE WEB INTERFACE ENABLES NON-EXPERT USE

Configuration• Gain & Exposure• Visual Control

Calibration• Intrisic and stereo calibration (factory setting)• Self-calibration during operation (automatic)• Hand-eye calibration by using wizards



• Holistic approach to perception and manipulation• Robustness and real-time capability• Consistent error estimation

(confidence, quality)• Low-cost approach• Versatile, modular and scalable• Easy setup and operation

Product Summary: Unique Selling PointsROBUST. REAL-TIME CAPABLE. COST-EFFICIENT. APPLICATION-ORIENTED.



AGRICULTUREINDUSTRIAL

AUTOMATION LOGISTICSSERVICE

ROBOTICS HEALTH CARE

crop identification, GPS-freenavigation, …

object recognition/ handling,…

autonomousnavigation, objecthandling,…

area monitoring, navigation,…

patientmonitoring, peopledetection,…

Application Domains



Appl

icat

ions

BIN

-PIC

KIN

G



Appl

icat

ions

TagD

etec

t, SL

AM, I

tem

Pick


Topic Group Perception coordination [SPARC]

Innovation Challenge

Research Partners


An Innovation PioneerA PARTNER IN RESEARCH PROGRAMMES AND COLLABORATIONS

THOMAS Mobile dual arm robotic workers with embeddedcognition for hybrid and dyna-mically reconfigurable manufacturingsystems [Proj. 723616/ EU Horizon 2020 FoF2-2016 programme]

FOR3D Schritthaltende 3D-Rekonstruktion und –Analyse [Bavarian Science Foundation (BFS)]

FORobotics Mobile, ad-hoc kooperierende Roboterteams [Bavarian Science Foundation (BFS)]

RoPHa Robuste Wahrnehmungsfähigkeiten für Roboter zur Unterstützung älterer Nutzer im häuslichen Umfeld [German Federal Ministry of Education and Research (BMBF)]



KUKA Udacity Robot Learning Lab at KITCROWD RESEARCH

Robot teaching, training, education and experimental laboratory for robotics students, professionals, and researchers.

Ten lightweight robot arms in a lab setup can be easily accessed online through the Internet and directly in the lab, and thus, used 24/7 around the world

• to learn the foundations of robotics,• to perform crowd research experiments, and• to offer robot competitions online.



Roboception GmbH

Kaflerstraße 281241 Munich

www.roboception.de

Sense. Reason. Act.on-demand.gputechconf.com/gtc-eu/2018/pdf/e8208...Dr. Michael Suppa I Co-Founder...

Documents

Transcript of Sense. Reason. Act.on-demand.gputechconf.com/gtc-eu/2018/pdf/e8208...Dr. Michael Suppa I Co-Founder...