BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0...

17
1 BRINGING AI-POWERED ROBOTS TO LIFE Deepu Talla, VP & GM – Autonomous Machines

Transcript of BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0...

Page 1: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

1

BRINGING AI-POWERED ROBOTS TO LIFEDeepu Talla, VP & GM – Autonomous Machines

Page 2: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

2

AI IS FUELING GLOBAL INDUSTRIESMulti-Trillion Dollar Global Industries Turning to AI

PUBLIC SAFETY

INDUSTRIAL

HEALTHCARE

TRANSPORTATION

SMART CITIES

STARTUPS

Page 3: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

3

ACCELERATED COMPUTING1000X EVERY 10 YEARS

1980 1990 2000 2010 2020

102

103

104

105

106

107

GPU PERFORMANCE

CPU PERFORMANCE+

Page 4: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

4

GRAPHICS HPC AI ROBOTICS

NVIDIA

HGX DRIVE & JETSONDGXRTX

Page 5: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

5

NVIDIA ACCELERATED COMPUTING

NVIDIA GPU CLOUD

CUDA-X

CUDA

HGX DRIVE & JETSONDGXRTX

GAMING

PRO VIZ

HPC

AI

TRANSPORTATION HEALTHCARE

ROBOTICS AI IOT

Page 6: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

6

ROBOTICS CONTROL LOOP

REASON& PLAN

PERCEPTION

ACT

Page 7: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

7

BRINGING AI ROBOTS TO LIFE

OPEN COMPUTING PLATFORM SOFTWARE & SIMULATION ECOSYSTEM

Page 8: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

8

JETSON NANO$99 NVIDIA CUDA-X AI Computer

CUDA-X acceleration stack | High-resolution sensor support | Runs all AI frameworks & models

Page 9: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

9

NVIDIA JETSON COMPUTING PLATFORM

developer.nvidia.com/jetson

CUDA | CUDA-X AI | Linux | RTOS | ROS 1&2

DEEPSTREAM

Module

sN

sight

Develo

per To

ols

ISAAC

JetP

ack

SD

K

DepthEstimation

ObjectDetection

PoseEstimation

GestureRecognition

PathPlanning

AutonomousNavigation

EcosystemModules

Deep Learning Computer Vision Accel. Computing Graphics Multimedia Sensors

TensorRTcuDNN

VisionWorksOpenCV

cuBLAScuFFT

VulkanOpenGL

LibargusVideo API

DriversEcosystem

Jetson Nano Jetson TX2 Jetson AGX Xavier

Page 10: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

10

CONTINUOUS SOFTWARE UPGRADES

MAR2016

SEPT2016

MAR2017

MAR2018

DEC2018

Jetpack 2.1/2.2

Jetson TX1

CUDA 7.0

cuDNN 4.0

Ubuntu 14.04

Kernel 3.10

Jetpack 2.3.x

CUDA 8.0

cuDNN 5.1

TensorRT 1.0

Jetpack 3.0/3.1

+ Jetson TX2

2x inference perf

cuDNN 6.0

TensorRT 2.1

Ubuntu 16.04

Kernel 4.4

Jetpack 3.2/3.3Deepstream 1.5

TensorFlow

CUDA 9.0

cuDNN 7.0

TensorRT 4.0

Jetpack 4.0/4.1Deepstream 3.0

+ Jetson Xavier

CUDA 10

TensorRT 5.0

Ubuntu 18.04

Kernel 4.9

MAR2019

Jetpack 4.2

+Jetson Nano

TrustedOS

Jetpack 4.2.1

Deepstream 4.0

ISAAC SDK 2019.2

Docker

Secure boot

JUN2019

Page 11: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

11

JETSON POWERING AUTONOMOUS MACHINES

WAREHOUSE DELIVERY AGRICULTURE INDUSTRIALRETAIL

Page 12: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

12

NVIDIA ISAAC

Isaac Robot Engine – Modular robot framework | Isaac Sim - Virtual robotics laboratory

Isaac Gym – Reinforcement learning simulator | Isaac Robot Apps – Kaya, Carter and Link

developer.nvidia.com/isaac-sdk

CARTER (Xavier)KAYA (Nano) LINK (Multi Xavier)

JETSON NANO

ISAAC OPEN TOOLBOX

Sensor and Actuator Drivers Core Libraries GEMS Reference DNN Tools

CUDA-X

Isaac Robot Engine

JETSON TX2 JETSON AGX XAVIER

Isaac Sim Isaac Gym

Page 13: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

13

SIMULATION

PhysXIsaac Sim

ISAAC SIM PHYSX

Page 14: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

14

NVIDIA ROBOTICS ECOSYSTEM

AI FRAMEWORKS

ROBOTICS SOFTWARE SOLUTIONS

SENSORS

REFERENCE PLATFORMS

ROBOT COMPUTERS

DEVICE MGMT & CLOUD INTEGRATION

AWS

Robomaker

Page 15: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

15

FIRST Robotics

Lockheed Martin AlphaPilot AI

Drone Innovation Challenge

RobonationDJI RoboMaster AI Challenge

RoboCup

ROBOTICS COMPETITION

Picking Challenges - Amazon, JDX

Page 16: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel

16

OPEN COMPUTING PLATFORM SOFTWARE & SIMULATION ECOSYSTEM

developer.nvidia.com/jetson

Page 17: BRINGING AI-POWERED ROBOTS TO LIFE...Deepstream 1.5 TensorFlow CUDA 9.0 cuDNN 7.0 TensorRT 4.0 Jetpack 4.0/4.1 Deepstream 3.0 + Jetson Xavier CUDA 10 TensorRT 5.0 Ubuntu 18.04 Kernel