Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI...
Transcript of Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI...
![Page 1: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/1.jpg)
Design GPU Systems for Hyperscalers ,Diverse AI Applications and Open Compute standard datacenters
Nick Yan
PDT Manager of AI Product Line of Inspur
![Page 2: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/2.jpg)
Inspur AI Market Share
Vertical AI Solutions
Caffe-MPI
AIStation T-Eye
GPUServer
CPUServer
FPGA Accelerator
End-to-End AI Solutions
Comprehensive Management Suite
Optimized Frameworks
Leading AI Computing Platform
TensorFlow-Opt
80%+
CSP
60%
IVA
55%
Telecom
80%
Finance
Inspur Full-Stack AI System
Inspur AI Server Growth
Global AI Server Growth
0
200
400
600
2017 2018
AI … 600%
150%
Inspur Radical AI Growth
Inspur is a leading cloud computing and AI computing data center infrastructure provider
Top 3 server vendor according to Gartner and IDC
AI full-stack solution provider
Design GPU Systems for versatile scenarios
![Page 3: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/3.jpg)
GTC2019· San Jose
NF5488M5
AI Training
4U 8x V100, NVSwitch
Industry - First AI Server
8 V100 GPU with NVSwitch Enabled
IPF2018· Beijing
NF5468M5
AI Cloud/Inference
4U 8x V100/4U 16x T4
Elastic GPU server
designed for AI cloud.
ISC2017 · Frankfurt
GX4
PCI-E Pooling
2U 4x GPU BOX
Flexible Expansion, available for
2-16 GPU cards extendibility.
GTC2019 · San Jose
NE5260M5
Edge AI
2U 2x V100/ 6x T4
Design for Edge Computing
End to End Computing AI Product Portfolio
SC 2018 · Colorado
AGX-5
AI Training
8U 16x V100, NVSwitch
World’s highest density 2U server of
8 highest performance GPUs.
HyperScaler New Edge Usage
![Page 4: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/4.jpg)
Nvidia’s HGXHigh Volume
Open Standard Motherboard World Class Reliable &
High Performance
Creating World’s Most Powerful & Reliable System
![Page 5: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/5.jpg)
Pushing the Envelop With HyperScaler
4 socket Platforms on Project Olympus
![Page 6: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/6.jpg)
NF5488M5
AI Training
4U 8x V100, NVSwitch
Industry - First AI Server
8 V100 GPU with NVSwitch Enabled
NF5468M5
AI Cloud/Inference
4U 8x V100/4U 16x T4
Elastic GPU server
designed for AI cloud.
AGX-2
AI Training
2U 8x V100/NVLINK
Minimum SizeMaximum Performance
NVIDIA® NVLink™ Enabled
.
NE5260M5
Edge AI
2U 2x V100 / 6x T4
Design for Edge Computing
End to End Computing AI Product Portfolio
AGX-5
AI Training
8U 16x V100, NVSwitch
World’s highest density 2U server of
8 highest performance GPUs.
HyperScaler New Edge Usage
![Page 7: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/7.jpg)
AGX-5The Most Powerful / Dense AI Server
AI Training Infrastructure AGX-5 Overview
HGX’s Wave “Zero” Partner Leading OEM partner to design HGX-2 Solution
Volume Ramp Choice by HyperScaler
Hyper Redundancy Design
Up to (2+2) *2 PSU Redundancy Design
Active parts are all Hot-swappable
8U with 850mm Depth
Up to 5x AGX-5 within 42U rack space
Proven Common Building Blocks (CBB)
Leverage High Volume Motherboard with
Nvidia’s HGX-2 to create an super reliable
system
![Page 8: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/8.jpg)
AI Training Infrastructure NF5488M5 Overview
NVIDIA® NVSwitch,2.4TB/s Aggregate Bandwidth
GPU-GPU bandwidth 300 GB/s
Full Speed on GPU-to-GPU communication
Best AC-DC Power Conversion Efficiency
Optimal Air cooling Efficiency
Build-in Server Node with NVMe DrivesFull function server node with 2x Xeon-SP with 3x UPI
Up to 8x NVMe SFF drives
Balance I/O Design NUMA balance I/O with 3x PCIe slot from each CPU
World Class Power & Cooling Efficiency
![Page 9: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/9.jpg)
AI Inference Infrastructure NF5468M5 Overview
Up to 20x PCIe x16 slots
World’s Dense Inferencing Server
HyperScaler Thermal QualityXeon Motherboard & GPU Board are Isolated to
to create an “non-shadow” thermal design
Design with Flexibility Support both V100 and T4
Each slots has full PCIe x16 bandwidth
Serviceability for Mass Deployment Most active components are design to be Hot-
swappable in order to reduce service downtime
![Page 10: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/10.jpg)
2U 8GPUs highest densityHigh Density
Minimum Size. Maximum Performance2U 8GPU Server with NVIDIA® NVLink™ Enabled
Superb Performance960 Tensor FLOPS, 376 TOPS on INT8.NVIDIA® NVLink™ 2.0 ready
Flexible Topology 10 Topologies of GPU for various applications.
High Speed ConnectionUp to 400G RDMA InfiniBand, optimized for low latency HPC, AI cluster
AI Training Infrastructure AGX-2 Overview
![Page 11: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/11.jpg)
Edge Application is Growing , AI included
CloudEdge
Edge
Edge
Edge
Edge
Edge
AutomotiveFinancial Service
Public Transport
Energy & Utilities
Manufacturing
Public Safety
Healthcare
Retail
Entertainment
Media
Agriculture
Logistics
![Page 12: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/12.jpg)
Edge AI Infrastructure NE5250M5&NE5260M5 Overview
Up to 2x V100 GPU card for Edge Training
Up to 6x T4 GPU cards for Edge Inferencing/Video Transcoding
World’s First Edge with GPU computation
430mm dept. , Front service-able
Super Compact Design for Rack and Edge
Uncompromised Xeon & Storage Support Support up to 2x Xeon-SP, 205Watt
16x DIMM slots
6x H/S SFF drive
Open & Application Focus
Compliant to OTII (Open Telecom IT Infrastructure) Perfect for NFVi, Composable Infrastructure
![Page 13: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/13.jpg)
6x T4 2x V100or
Flexible Edge Work On-Demand
![Page 14: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/14.jpg)
Market Leadership in GPU-focus System Design
HyperScaler Design Capability
High Performance & Most Reliable Systems
Pushing AI computation with 4 Socket Motherboard
End to End Computation – From Data Center to Edge
![Page 15: Design GPU Systems for Hyperscalers ,Diverse AI ... Y… · Inspur AI Market Share Vertical AI Solutions Caffe-MPI AIStation T-Eye GPU Server CPU Server FPGA Accelerator End-to-End](https://reader034.fdocuments.in/reader034/viewer/2022042807/5f813d5a06974768e15e9ad7/html5/thumbnails/15.jpg)
Thank You!