Inspur OAI Reference System

YJ Huang, Development Manager, Inspur

OAI

UBB Common Tray

OAM with Heatsink

Inspur Fab2 System

Compute Node: ON5263M5

UBB reference System: MX1

High DensityWhisper Cable

三視圖

141mm

Inspur System Overview

Front View

Rear View

Front IO connector QSFP-DD Connector for OAM Expansion

Whisper ConnectorHHHL PCIe Expansion

54V BUSBAR Clips 9256 Fan

Product model: MX1

Chassis 21” 3OU Rack mount

Dimensions 537W*141H*803D (mm)

Connection with Compute node Up to PCIe Gen4 x32

OAMSupport Max 8pcs 48~54V OAM(up to 450W each);Support Max 8pcs 12V OAM (up to 350W each)

Power without OAM 1570W

PCIe Switch Support PCIe Gen4 (100lanes/chip)

PCIe re-timer Support PCIe Gen4 x16

Phy re-timer 56Gbps PAM-4 or 10/28Gbps NRZ x16

Expansion slots Up to 4 x PCle Gen4 x16 low profile standard card

BMC AST2520

I/OUSB port for UART debug, RJ45 port for BMC dedicated, UID, PWR Button

Ambient Working Temperature 5-35 ℃

Inspur OAI System Features

System Explode View Top cover

PDB

HIBUBB

9256 Fan

Side cover

FIO

UBB Common Tray

Side cover

Air Duct

System Block DiagramCompute

NodeHIB

BMC AST2520

(Management)

PCIe Switch PM42100

PCIe Switch PM42100

PCIe Switch PM42100

PCIe Switch PM42100

CPLD

IB Module

UBB

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

PCIeRe-timerPT4161L

OAM0 OAM1 OAM2 OAM3 OAM4 OAM5 OAM6 OAM7

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

QS

FP-

DD

IB Module

Signal symbol Signal Type

PCIe x16

Management

Serdes

I2C

PDB

Power Monitor

I2C

I2C

54V to 12VVR x6

54V HSC x9

I2C/JTAG

CPLD

CPLD

PhyRe-timer

CRT50216P

PhyRe-timer

CRT50216P

PhyRe-timer

CRT50216P

PhyRe-timer

CRT50216P

System Placement

OAM6

OAM5

OAM2

OAM1

OAM7

OAM4

OAM3

OAM0

UBB HIB PDB

9256 Fan x4

Power Clip

QSFP-DD x8

Whisper conn. to IB

Whisper conn. to Compute node

PCIe x16 Conn. x4

Whisper Cable

FIO

IB Card

Universal Baseboard Placement

585mm

417mm

QSFP-DD Connector Amphenol UE36-A 1070-3000T

QSFP-DD Cage Amphenol UE36-B 16221-06A5A

12V Connector Amphenol JX410-513xx

54V Connector Amphenol 10028917-001LF

Guide Pin Amphenol 10037909-101LF

High Density Connector

Amphenol 10131762-101LF

Mezzanine Connector Molex 2093111115

High-speed Interface Board Placement

528mm

188.4mm

12V Connector Amphenol JX412-50432

54V Connector Amphenol 10052620-5544P00LF

Guide Pin Amphenol 10044314-101LFC

High Density Connector

Amphenol 10124558-101LF

Whisper Conn. to compute node

Whisper Conn. to IB

Power Conn. to PDB

System Access Solution – UBB tray

Press button to release lever handle.

Then pull out UBB tray from system.

OAM Heatsink Design For Common Tray✓Add top cover & HS module handle for accessing✓Reduce fin numbers to meet common UBB tray specification

Preliminary version Current version

Fin area height63.3mm

Fin area height 66.1mm

OAM x4HS: detail model

Switch x4

OAM x4HS: compact model

VR x6

QSFP-DD x8

Vent: 40%

Power Deliver Board PlacementThermal Design Power

Component Unit Power Q’ty Power (W)

OAM 450 8 3600

Re-timer 9.6 8 76.8

Switch 44 4 176

PHY 18.7 4 74.8

QSFP-DD 12.5 8 100

CX5 18.1 4 72.4

VR 25 6 150

9256 Fan 140 4 560

Total Power (watt) 4810

Re-timer x8

PHY x4

Top view

CX5 x4

Simulation Results (Temperature Field)Simulation Boundary Condition Setup

✓ TDP at 35C ambient

✓ Fan normal

✓ Sea-level

Cut plane: y=17.5mm

Airflow

Simulation Result

Location Spec (C) Temp.

OAMASIC 115.0 76.5

HBM 95.0 93.3

PCIe Re-timer 85.0 79.5

PCIe Switch 115.0 90.8

PHY Re-timer 125.0 78.7

QSFP-DD 70.0 66.9

CX5 105.0 94.1

VR 125.0 108.7

✓ Compact model for upper stream OAM HS

✓ Compact model HS for VR

✓ Symmetric placement for switch HS two for

detail model; two for compact model

Inspur 2OU Compute Node

Re-timer card with Whisper connector

2.5” HDDx4

8x10 Whisper Cable

Hea

d n

od

e 1

Hea

d n

od

e 2

54V->12VVR

54V->12VVR

Compute Node Explode View

Rack Level Solution

Cable symbol Cable Type

Whisper Cable (PCIe)

Eth cable

QSFP-DD (Serdes)

48VDC Open Rack

✓ 1 pairs 48V Bus Bar

✓ 1 shelf per Rack

Power Shelf

✓ 33KW(12xPSU)

✓ 40V-58V

✓ 93mm (H, 2OU) x 537mm (W, 21”) x 586 (D) mm

System Devices

✓ Inspur 3OU OAI systems x4

✓ Inspur 2OU compute node x4

100G Switch

OAI system

Compute node x2

OAI system

Power Shelf

OAI system

Compute node x2

OAI system

OAM Vendor

✓ Habana Gaudi

✓ Nvidia

✓ AMD

…etc

System Power Subtotal

Inspur OAI system20kW

(5kW x4)22.8kW

Inspur Compute Node2.8kW

(700W x4)

Total Power/Rack 33kW

1G Mgmt Switch

Call to Action

• Power on sample is already available @TWN office• Have Inspur reference system sample review in Experience lab

online booth• Project Wiki with the latest specification :

https://www.opencompute.org/wiki/Server/OAI• Mailing list:

https://ocp-all.groups.io/g/OCP-OAIV