A New Era of ComputeBRAND NEW IP: Pushing AI Compute Performance Ethos-N57 and Ethos-N37 NPUs ML...

© 2019 Arm Limited© 2019 Arm Limited

A New Era of Compute

Ian SmytheVP Marketing for Client, Arm


The Fifth Wave of Computing

IoT

Data

5G+

Edge Compute

AI

Changing our world forever.

© 2019 Arm Limited

New Experiences are New Opportunities

© 2019 Arm Limited

Immersive Experiences

Domain Specific vs General Purpose

Compute

Device Innovation

5G-ready Platforms

Rise of Artificial Intelligence

Automotive

© 2019 Arm Limited

ı

Total Compute: Task-specific SoC Customization

Shifting from a product evolution focus to use-case driven solutions

Optimizing across IP boundaries as well as evolving products within the solution

Adopting a system approach to solve for the workloads of tomorrow

© 2019 Arm Limited

© 2019 Arm Limited

The Foundation of Total Compute

Compute Performance:

A solution-based approach for accelerating performance growth

Security: Creating a secure foundation through hardware and software

Software and Tools:

Making the performance more accessible for developers


Evolving CPU Compute Performance

Dot product:Cortex-A75/A55• Extra Neon pipeline• Compute libraries

MatMul:

Matterhorn• New instructions

for matrix multiply

Today Tomorrow


Rapidly-increasing CPU ML Performance

MatterhornCortex-A73 Cortex-A76 Cortex-A77 HerculesCortex-A75

MatMul

Big Core GEMM Performance

Dot Product

Implementation performance may vary

~10x

© 2019 Arm Limited




Software and Tools:



© 2019 Arm Limited

Security is Everything Their Digital Life is in Our Hands

Who you are, and what you do

Identity

Digital Wallet

Location

Passwords Social Media Account

Family andFriends Contacts

Biometric Identifiers Bank and FinanceApps

Medical Information

Always connected

© 2019 Arm Limited

Enabling PSA-Client to create easier to maintain security

implementations

Strong protection against return and

jump-oriented exploits

Detect memory safety vulnerabilities

C

Secure-EL2

PointerAuthenticationand Branch Target Identifiers

Memory Tagging

Delivering Security Features the Ecosystem Needs

© 2019 Arm Limited




Software and Tools:



© 2019 Arm Limited

Building an Ecosystem Under a Common Architecture

23 Million Software Developers

Source: Evans Data Corporation, 2018

Total Compute tools unleash greater performance from silicon

Virtual prototypes reduce risk and time to market

Software developers have fewer tools to learn

© 2019 Arm Limited

© 2019 Arm Limited

23 Million Software Developers

Source: Evans Data Corporation, 2018

© 2019 Arm Limited

Total Compute devices become the platform of choice for developers

Building an Ecosystem Under a Common Architecture

© 2019 Arm Limited

• Enable Unity creators to improve Arm silicon performance in their native environment

• Further optimize the Unity engine to get even better performance across Arm CPUs, GPUs and NPUs

Games Developers: Take Note!Making it easier for content creators and developers


Made with Unity


Fortnite World Cup Sees a 16-year-old Win Record$3 Million…

© 2019 Arm Limited

Cortex-A77

Redefining Mobile performance

Mali-G77

New architecture, incredible HD gaming

Mali-D77

Superior display technology for VR

2019 Premium Mobile SolutionsPremium IP launched May 2019

© 2019 Arm Limited

BRAND NEW IP: Pushing AI Compute Performance

Ethos-N57 and Ethos-N37 NPUs ML inference processors for vision, voice and more

for home and mobile markets

NEW Mid-range NPUs

Ethos-N77 NPUPerformance-critical AI applications

delivering premium experiences

(Formerly ‘ML Processor’)

© 2019 Arm Limited

Arm Ethos NPU Family Enables Multiple IP Choices for Devices

SmartCameras

EntrySmartphones

DTVMainstream

SmartphonesSmart

Home HubComputationalPhotography

PremiumSmartphones AR/VR

Ethos-N37

Ethos-N57

Ethos-N77

4 TOP/s

2 TOP/s

1 TOP/s

< 2 GB/S 4 GB/S 8 GB/S

© 2019 Arm Limited

Open-source Standard Software Speeds ML Deployment

Parsers

Model conversion and optimization tooling

Application

Network import(eg. TensorFlow, Caffe, ONNX)

Connect to inferenceengines

Training-timetooling

NN API

NN inference engine

Seamless dispatch todedicated IP

Optimized NN algorithms

3rd party IP

NPUs CPUs GPUs

Pro

filin

g an

d d

ebu

ggin

g to

olin

g

Connect to high-level

frameworks

1

1

Supported byend-to-end

tooling

4

4

Integrateadditional IP

3

3

Connect toinferenceengines

22

© 2019 Arm Limited

NEW Graphics and Display Processors Filling Your Life with Color

Mali-D37 DPU The most energy-efficient,

smallest area display processor for Full HD and 2K resolution

Mali-G57 GPU 1st mainstream Valhall-based graphics

processor for high-resolution UI in 4K/8K DTV, AR and high-fidelity gaming


Leap in Gaming Performance and EfficiencyEfficiently supporting growing graphics and ML complexity

Compared to Mali-G52 3EE on same process node under similar conditions

30%Better energy

efficiency

30%More

performancedensity

60%Improvementfor machine

learning

© 2019 Arm Limited

First Komeda Architecture DPU for Mainstream Market

30%System power savings and memory

management

Compared to running the same operations on the GPU

Delivers 2K and Full HD In Smallest Area of <1mm2

© 2019 Arm Limited

CloudData Centers

Edge

EdgeCritical Data

Massive Amounts of Data

Trillionsof Devices

z

Edge

5G z

Cloud to Edge Infrastructurefor 1TN Devices

© 2019 Arm Limited

AI Edge

To

Network BridgeFrom

Network Function Management

Network Prediction

Network Function Management

Application Decision-making

AI Centric

Cloud-native

DevOps Managed

Multi-tenancy & Virtualized

Secured

Evolution of the Edge

© 2019 Arm Limited

Project CassiniEnsuring a cloud-native experience across a diverse and secure edge ecosystem

HW Acceleration

OS / Containers / Hypervisor

Arm Neoverse Platform

Root of Trust

RU

NTI

ME

SEC

UR

ITY

SER

VIC

ES

Net

wo

rk S

ervi

ces

SQL/

No

SQL

Dat

a En

gin

es

An

alyt

ics

Filt

er /

Rea

ct /

Sto

re

Arm PSA

Net

wo

rk P

red

icti

on

Edge

Sta

ck

Platform standards & reference systems

Cloud-native software stack

PSA extended for a secure infrastructure edge

© 2019 Arm Limited30 Arm Limited © 2019

Navigating the Road to Autonomous Vehicles

MCU Sensors

Cockpit

L4/5 AVPerception

© 2019 Arm Limited

© 2019 Arm Limited

Autonomous Vehicles: The Fourth Missing Link Collaboration

• Automotive and technology industry leaders coming together

• Helping to accelerate the delivery of safe autonomous vehicles at scale

• Enabling a common compute architecture for autonomous driving

• Publishing results of the working groups for the benefit of all

© 2019 Arm Limited


Cortex-M33With Garda

Arm Custom Instructions: Do Even More to Differentiate on Arm

The right-sized compute everywhereArm NPU

CustomAccelerator

Cortex-M33

Co-pro

Co-pro

Cortex-M33 in early 2020, built into all future M-class

Standard software development environment

Custom instructions interleaved with Arm standard instructions


A Simple Path to Customization

Pre-designed control provided

by Arm

Writeback

Partner custom execution logic

Standard execution logic

Execute

Custom instruction space

Standard instruction space

DecodeFetch

Pre-designed control provided

by Arm

Arm Architecture (Armv8-M)Security, functional safety, software ecosystem

CPU

© 2019 Arm Limited

Arm Custom Instructions

Standard software development environment

Custom instructions interleaved with Arm standard instructions

First core: Cortex-M33 in early 2020, and then built into all future M-class

Software Ecosystem-Friendly

AP

Is Optimized algorithms

Armv8-M ISA

Build with standard compiler

Application code

Custom Instructions

Cost-very friendly ($0)

37 © 2019 Arm Limited

Enabling Experiences Everywhere

Immersive Experiences for Everyone

AI at the edge

EdgeData Centers

Project Cassini

Total Compute

Arm CustomInstructions

Autonomous Vehicles


Thank you

A New Era of ComputeBRAND NEW IP: Pushing AI Compute Performance Ethos-N57 and Ethos-N37 NPUs ML...

Documents

Transcript of A New Era of ComputeBRAND NEW IP: Pushing AI Compute Performance Ethos-N57 and Ethos-N37 NPUs ML...