Robotic Design: Frontiers in Visual and Tactile Sensing

Thank You To Our Sponsors

q  This webinar will be available afterwards at www.designworldonline.com & via email

q  Q&A at the end of the presentation q  Hashtag for this webinar: #DWwebinar

Before We Start

Moderator Presenters

Paul Heney Design World

Gerald Loeb Biomed Concepts

SynTouch

Goksel Dedeoglu PercepTonic

Robotic Design: Frontiers in Visual and Tactile Sensing

presented at the

December 4, 2014

Goksel Dedeoglu, Ph.D. Founder, PercepTonic, LLC

Robotics Then…

Moravec,  1980

SRI’s  Shakey,  1967 The perception needs of robots have motivated much of the research in

Computer Vision over the past 50 years.

©  PercepTonic,  LLC

Moravec,  1980

SRI’s  Shakey,  1967

thanks  to   50  years of  consumer   electronics

Pelican  Imaging

HTC  M8

Subaru

XBOX  Kinect

Robotics Today

Prox  Dynamics

MAST

Brain  Corp.

©  PercepTonic,  LLC

The Promise of Computer Vision Computer Vision has advanced from factory-floor

automation to real-world problem solving in unconstrained environments

Embedded Ever-Ready Everywhere

©  PercepTonic,  LLC

Computer Vision: Images In, Information Out LE

NS

•     Size  and  shape •     3-­‐‑D  position  &  orientation •     Identity,  object  class •     Expression,  gesture •     Location •     Motion •     Illumination,  weather,  …

INFORMATION CAMERAS

Computer Vision algorithms analyze images to extract information about the world.

This enables machines that can perceive.

©  PercepTonic,  LLC

How Does a Robot “See”?

Contextual understanding: Who, What, Where? Real-time operation with visual feedback (do until …) Learning and adaptation

TeraDeep

OMRON

©  PercepTonic,  LLC

Tools That Help Robots “See” Vision algorithms customized for tracking, segmentation and recognition Power- and cost-optimized processors with expert partitioning between edge and cloud loads Sensors attuned to the particular needs of the computer vision task Intel

OmniVision

©  PercepTonic,  LLC

Case Study: 3D Depth Perception Example: Subaru’s EyeSight

©  PercepTonic,  LLC

How Stereo Vision Creates 3D Depth Perception

•  Works by triangulation: the closer an object is, the greater its

parallax between the left-right views.

•  Stereo is a passive depth sensing method. Active methods include structured light (Kinect) and Time-of-Flight sensors.

left  image right  image

+ disparity  image (inverse  depth)

   stereo      algorithm

output input

©  PercepTonic,  LLC

Application: Automotive Safety Obstacles detected via stereo depth Stereo Depth/Disparity output

Source:  Texas  Instruments for  more  information,  see  hap://goksel-­‐‑dedeoglu.com/3d-­‐‑stereo-­‐‑vision.html  

©  PercepTonic,  LLC

Challenges in Embedded Vision

Computer Vision has the potential to fuel the next wave of smart and affordable robots

but…

•  The algorithms are pixel-intensive and computationally expensive

•  The solutions are still evolving –new algorithms every year!

•  The processors and sensors are really optimized for multimedia

©  PercepTonic,  LLC

Embedded Vision Architectures Our best guess for power-efficient vision computation is

heterogeneous and multi-core architectures.

•  Mobile SoCs: Qualcomm Snapdragon, NVIDIA Tegra K1

•  Established DSPs and FPGAs, Movidius’ Myriad

•  Vision IP: CogniVue, videantis, CEVA, adapteva

•  Early examples of fixed-function hardware for vision

•  Power-efficient server design starting to adopt the same typical  block  diagram   of  modern  SoCs

©  PercepTonic,  LLC

The Future of Computer Vision: Embedded, Ever-ready, and Everywhere From    proof-­‐‑of-­‐‑concept    prototypes    to    products

“PC  World” unlimited  CPU,  GPU,  …

“Embedded  World” select  processors  &  sensors

“Product  World” finalized  hardware

©  PercepTonic,  LLC

We Have Come a Long Way…

Moravec,  1980

SRI’s  Shakey,  1967 Consumer Electronics is now driving significant investments in computer vision HW and SW,

helping the robotics vision wish-list come true.

©  PercepTonic,  LLC

References and Resources Case studies and tutorials at the PercepTonic web site

http://www.PercepTonic.com/case-studies.html

The Embedded Vision Alliance (2011-present) http://www.embedded-vision.com

http://embeddedvisionsummit.com

The IEEE Embedded Vision Workshops (2005-present) http://cvisioncentral.com/evw2014/

©  PercepTonic,  LLC

Thank You! slides will be available at

http://www.PercepTonic.com/slides/DesignWorld

©  PercepTonic,  LLC

The Future of Machine Touch®

Gerald  E.  Loeb,  M.D.,  CEO

Touch is essential for dexterity and perception.

Simple tasks become impossible without it.

The Absence of Machine Touch

Precise Expensive Insensate

Rigid Dangerous

Soft

Sensitive Adaptive

•  Works in unstructured environs

•  Safe near humans

•  Performs a variety of tasks

•  Does not need retooling

Robots Doing Human Jobs Need Biomimetic Design

BioTac® Biomimetic Tactile Sensors Supple Fingertips

Multi-Modal Sensing

High Bandwidth

Durable

Field Repairable

Scalable

Live  LabView  GUI  Demo

EPFL Lausanne Allegro Hand

Kuka Arm BioTacs

Dexterous Autonomous

Robots

The Today Show: Popular Mechanics Breakthrough Award

Contact Detection Reflex Enables Dexterous Use of a Myoelectric Prosthetic Hand

Blaine Matulevich, Vikram Pandit, Jeremy Fishel

Biomimetic Inhibitory Reflex for

Fragile Grasp

0 1 2 3 4 5 60

2

4

6

8

10

Preamplified Net EMG Signal (V)

Mot

or C

omm

and

(V)

ContactNo Contact

Performance

on cracker task Speed

Accuracy

error

Part Misalignment Failures

Contact Forces

Ejection

Displacement

Part Misalignment Failures

Contact Detection

Solution with Tactile Feedback

NumaTac® “airbags for robots”

Companies Want to Understand Consumer’s Haptic Perception of Their Products

Focus Groups and Sensory Advisory Panels are expensive, slow and prone to “noise”.

Product releases have $Billions at stake.

BioTac texture robot provides objective data about human

dimensions of product “feel”.

Bayesian Action&Perception: Representing the World in the Brain

G.E. Loeb and J.A. Fishel

Frontiers in Neuroscience

vol. 8 (October, 2014)

Tactile Sensing for Humans-in-the-Loop: Telerobots for Hazardous Environments

•  Tactile sensing is essential in low visibility environments.

•  Fast reflex responses are more useful than conscious perception.

•  Government Applications for Dexterous Telerobots: Ø  DOD – ordnance

Ø  DARPA – drones

Ø  NASA – space

Ø  NOAA – underwater

Ø  NRC – nuclear reactors

“The Sense of Touch”

Jusepe di Ribera

ca. 1615

Norton Simon Gallery, Pasadena, CA

The Standard in Machine Touch® www.SynTouchLLC.com

Willow  Garage  PR2 Allegro  Hand Schunk  Dexterous  Hand Shadow  Hand

HUBO  Robot Robotiq  Gripper Barrea  Hand Kinova  JACO  Arm

Questions? Paul Heney Design World pheney@wtwhmedia.com Twitter: @DW_Editor

Goksel Dedeoglu PercepTonic goksel@perceptonic.com Phone: 214.356.4640

Gerald Loeb SynTouch/Biomed Concepts geloeb@gmail.com Phone: 213.944.2283

Thank You q  This webinar will be available at

designworldonline.com & email

q  Tweet with hashtag #DWwebinar

q  Connect with Design World

q  Discuss this on EngineeringExchange.com