Generalized Cloud and Edge Compute Architecture for AIoT · 29/05/2019 · Super Micro Computer,...

© 2019 Supermicro ®

We Keep IT Green™

Generalized Cloud and Edge Compute Architecture for AIoT

Michael Clegg - VP and GM IoT / Embedded

Computex 29 May 2019


SUPERMICRO CONFIDENTIAL

3

.

0



3

.

0

What did we do with 100M transistors ?

Software

Graphics

Audio

Storage

Video

Hardware

Graphics

Audio

Storage

Video



3

.

0

Some Things Replaced by Smartphone

And counting … “..50 Things Your Smartphone Replaced [ Or Will Replace In The Future ]



3

.

0

Moore’s Law Drives Virtualization

Processor Cores

Memory

Storage

Required

Available

Capacity Multi-Core CPU

Hypervisor

Container Container Container

App App App

Intel®



3

.

0

All-IP Simplifies Networks

PSTN

LL

ATM

FR

X.25



3

.

0

Which Leads to Virtual CPE

Intel® Certified uCPE hardware and software with a choice of apps

MANO Cloud

NFVI

Hardware

Ensemble Connector

Network OS NFVI

Software

VNFs

SYS-1019D-16C-FHN13TP

Firewall App App SD-WAN



3

.

0

Supermicro Telco Edge Platform

Computing

Up to 16 Cores / 32 Thread / SoC / Socket CPU / GPU

Network

Up to 37 Ports / SFP+ / 10G Base-T / 1GbE

Storage

Flexible form factor / M.2 / 2.5”/ EDSFF

Power Input

Redundant PSU / DC / AC / Power Adapter



3

.

0

Telecom - 5G Built on COTS

Public Cloud

Data Center

Container

VNF

COTS

Container

VNF

COTS

Container

VNF

COTS

Container

VNF

COTS

Container

VNF

COTS

COTS – Commercial Off The Shelf (x86 Server)



3

.

0

1980 1990 2000 2010 2020

Mainframe & Terminal

Client-Server (PCs)

Mobile and Cloud

5G & IOT / Edge (Immersive)

IT architecture progression correlates to AI development milestones

(rule-based, robots, cognitive computing, deep learning…)

ML: training in CLOUD, inference at Edge

AI/ML facilitates the rise of the new generation IT architecture

2030

AI/ML Transforming IT Infrastructure

What About AI / ML



3

.

0

AI : Training Not Programming

(Reinforcement Learning)



3

.

0

Machine Learning Process Deep Learning

BIG DATA Collecting and preparing

training information from

Data Mining and Analytics

TRAINING Learning a new

capability from existing

data (training materials)

INFERENCE Applying this

capability to new data

(input)



3

.

0

Technology Building Blocks for AI/ML

Accelerator

(GPU / PFGA…)

Storage Compute Networking

Persistent Memory

& All-flash NVMe

1G / 10G / 25G / 100G / 200G

BIG DATA Collecting and preparing

training information from

Data Mining and Analytics

TRAINING Learning a new

capability from existing

data (training materials)

INFERENCE Applying this

capability to new

data (input)

Deep Learning



3

.

0

Big Data – Training Materials

Turnkey

Distributed file system

MapReduce

Scaling out architecture

Turkey solution

Established technology

Combining with AI

implementation

Deep Learning



3

.

0

Training in Cloud – Scalability

Non-blocking Interconnect

(E.g. FatTree topology)

Scaling-out Scaling-up

The Most Powerful Training

Machine (HGX-II)

Delivering 2 PetaFLOPs of DL

performance (TOPs)

Deep Learning



3

.

0

Inferencing & Edge Computing

** FPGA’s are sometimes used in the Edge server

to optimize the Inference performance

Deep Learning



3

.

0

GPU Optimized Servers for ML (examples)

4029GP-TRT2 1029GQ-TVRT

DP SKYLAKE CPU; 3UPI

12 DDR4 DIMMs

2 HS HDD bays

4 SXM w/ NVLink2

4 x16 PCIe 3.0 slot

2 2000W Titanium PWS

Scalability


24 DDR4 DIMMs

24 HS NVMe HDD bays

10 Double-Wide devices

12 x16 PCIe 3.0 slot;

4 (2+2) 2000W Titanium PWS

Flexibility

4029GP-TVRT


24 DDR4 DIMMs

16 HS HDD bays (w/ NVMe)

8 Pascal w/ NVLink2

6 x16 PCIe 3.0 slot

4 (2+2) 2000W Titanium PWS

HyperScale

Design

Topology

E.g. CNN: Convolution Neural Network

Deep Learning



3

.

0


We Keep IT Green™

IoT Edge Computing, AI and 5G

Mory Lin

Sr. Director, IoT / Embedded Products

Architecture Department, Supermicro

05/29/2019

Product Design Architecture



3

.

0

Edge Computing Optimization

Computing 16 ~ 28 Cores

Network 10GbE /SFP+

Accelerator FPGA / GPU

Storage NVMe/ EDSFF/ M.2

Compact 1U / Box / Fanless

Outdoor IP65 / IP67

Operating Temp.

Long Life 7 Years +

Virtualization Open

Architecture

Orchestration Flexibility Multiple Cards

Edge Computing Solutions

28-Core Xeon Scalable/

16-Core Xeon D

2x EDSFF Short SSDs

Up to total 37 LAN ports

10G/25G/50G/100G

15” Depth / 45°C

VNF SD-WAN uCPE C-RAN

19

AI Inferencing



3

.

0

20

Edge Computing Infrastructure

Edge Computing

Security

Analytics

AI

Devices Management

Virtualization

Orchestration

Network

Storage

GPU

FPGA

QAT

Video Analytics

Healthcare

IT

Retail

Intelligence Industry 4.0

Software

Defined

Network

5G IoT



3

.

0

Edge Computing Platform

21

Open Platform / Low Power / Scalable Architecture

Fanless Box PC 1U / Box Sever

/ 1 Slot

1U Sever /

1 Slot /

2x Hot-Swap HDD

1U Sever /

2 Slots /

2x Hot-Swap HDD

2.5U Sever /

3 Slots /

4x 2.5” HDD

Mini Tower/

1 Slot /

4x 3.5” HDD

1U / 4 Slots /

Redundant PSU

Atom

Core i Atom

Core i

Atom

C3000

Xeon

E-2100

Xeon

D-2100 Xeon

Cascade Lake-SP

5-15W 35W~45W Up to 100W Up to 205W



3

.

0

Supermicro Xeon Scalable Processor and Xeon D Solution for Edge Computing

IoT Edge Computing

AI Vision Computing

GPU Inference

Workload Consolidation

Accelerated Computing

Network Transformation

Support 28/16 Cores

Support GPU card

Support FPGA card

Support 2 cards in 1U

Support up to 37 LANs

Compact Edge Systems

Software

Applications

Hardware

Designs

Best Known Configurations



3

.

0

2nd Gen. Intel® Xeon® Scalable processor (Cascade Lake) and Xeon D (Skylake-D)

23

SYS-E403-9P-FN2T, up to 28-Core Wall-mount System with 3 PCIe slots

SYS-1019P-FHN2T, up to 28-Core 1U Compact System with 2 PCIe slots

High Performance Edge Computing Platform New!

SYS-E403-9D-16C-FN13TP , 16-Core Wall-mount System with 3 PCIe slots

SYS-1019D-16C-FHN13TP, 16-Core 1U Compact System with 2 PCIe slots

SYS-1019D-16C-FRN5TP, 16-Core 1U Compact System with 4 AIOM slots



3

.

0

24

Atom ® and Core ® i7/i5/i3 Smart Fanless Gateway

SYS-E100-9AP

SYS-E100-9APP

SYS-E100-9S

SYS-E100-9S-E

SYS-E100-9S-L

SYS-E100-9AP-IA

SYS-E50-9AP

SYS-E50-9AP-Wifi

SYS-E50-9AP-N5

SYS-E50-9AP-L

• Palm Size/

• Built-in Antenna

• Slim Design

• Wide Operating Temp

• Performance

• vPro / iAMT (Core i7/i5)

• 9-36V DC

• DIN Rail mounting

• I/O Expansion



3

.

0

25

AMD Embedded EPYC 3000 SoC


We Keep IT Green™

AIoT Applications

26



3

.

0

Smart Factory http://ima-net.org/smart-factories-could-add-500-billion-to-global-economy-in-5-years/



3

.

0

Smarter Medical Expert Systems https://thenextweb.com/artificial-intelligence/2019/05/01/ai-is-already-changing-how-cancer-is-diagnosed/



3

.

0

Smart Retail Applications

https://www.adweek.com/digital/walmarts-intelligent-retail-lab-will-test-ai-in-brick-and-mortar/



3

.

0

How Many Cars?

Taking Advantage of Opensource / Trained Models

5G Drone Applications



3

.

0

Drone-Based Object Counting: Spatial + Appearance Constraints

demo

Hsieh et al. Drone-Based Object Counting by Spatially Regularized Regional Proposal Network. ICCV 2017

Video goes here



3

.

0

Taking Advantage of Opensource / Trained Models

Possible Applications - 5G enabled



3

.

0

Thank You

For more information, visit www.supermicro.com or contact

[email protected]

WWW.SUPERMICRO.COM



3

.

0

Disclaimer

Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The

information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions

and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate

performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware

configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation

of third party benchmarks or websites referenced in this document. The information contained herein is subject to change

and may be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap,

component and hardware revision changes, new model and/or product releases, software changes, firmware changes, or

the like. Super Micro Computer, Inc. assumes no obligation to update or otherwise correct or revise this information.

SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE

CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS

THAT MAY APPEAR IN THIS INFORMATION.

SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY

OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO

ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE

USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY

ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

ATTRIBUTION

© 2019 Super Micro Computer, Inc. All rights reserved.

Generalized Cloud and Edge Compute Architecture for AIoT · 29/05/2019 · Super Micro Computer,...

Documents

Transcript of Generalized Cloud and Edge Compute Architecture for AIoT · 29/05/2019 · Super Micro Computer,...