Enterprise class 64-bit ARM server update LinuxCon …...Enterprise class 64-bit ARM server update...

Enterprise class 64-bit ARM server updateLinuxCon Japan 2016Standardization, Ecosystem enablement, and Red Hat Enterprise Linux

Jon MastersChief ARM Architect and RHEL Server for ARM Technical Lead, Red Hat

3

Embedded ARM board running Fedora

64-bit ARM Server running Red HatEnterprise Linux Server for ARM

Development Preview

Enterprise class 64-bit ARM serversA five year journey from embedded to real server-class Enterprise systems

4

Five years of “firsts” with 64-bit ARM serversRed Hat began work on ARMv8 five years ago and has been involved with every design

First ARMv8server hardware

First ARMv8 servercluster (also ceph)

First Apache Sparkanalytics cluster

5

Why are 64-bit ARM servers interesting?

• Large “hyperscale” cloud companies want alternatives• Reduces cost, promotes competition, aids design flexibility

• Future designs more focused on IO, memory, compute offload• Single core compute performance potential is peaking• ARM vendors integrate differentiating accelerators

A combination of factors in the industry and market make ARM servers viable

6

Why are 64-bit ARM servers interesting?

• APAC regional mandate for independent technology leadership• China market in particular wants “home grown” options

• Slowing pace of Moore's Law provides disruptive potential• Future ARM processors will match performance of

contemporary computer architecture options

A combination of factors in the industry and market make ARM servers viable

7

Where can I get a 64-bit ARM server system?“Tier 1” Cloud vendors often implement their own custom hardware platforms but you can also get a range of OEM platforms from traditional and emerging suppliers.

Penguin Computing Valkre series (2 socket Cavium ThunderX)

8

Where can I get a 64-bit ARM server system?“Tier 1” Cloud vendors often implement their own custom hardware platforms but you can also get a range of OEM platforms from traditional and emerging suppliers.

HPE Moonshot Proliant m400 cartridge (AppliedMicro X-Gene)

9

BUT...Where can I get a 64-bit ARM server?!!Two examples of “low cost” platforms featuring AMD's “Seattle” A11xx - for those who aren't “Tier 1” cloud vendors buying their ARM servers in “hyperscale” dense rack form factors

96Boards Enterprise EditionLeMaker “Cello” board ($300)

SoftIron OverDrive 1000 ($600)A full system in a case with HDD

10

What is an ARM server system anyway?

• 64-bit ARMv8.x-A class SoCs featuring up to 10,000s of cores• Dozens of different silicon vendors (licensed vs. custom)• Differentiation comes in performance, SoC devices, etc.

• Built to common industry standards• Server Base System Architecture• Server Base Boot Requirements

Some definitions to help explain the focus of our development activities

11

64-bit ARMv8.x System-on-Chips

• Server Base System Architecture lays the “ground rules”• Different levels of system implementation (level 0-2...)

• Multiple 64-bit ARMv8.x-A coherent cores (e.g. 4-52 cores)• Licensed from ARM: Cortex-A53,57,72…• Custom implementation: Potenza, Vulcan, Thunder…

What is in an ARMv8 64-bit server class SoC?

12


• Common base IP blocks required for a complete system• Standard UART, Interrupt controller (GICv3), SMMUv3…

• Standard I/O and interconnects (SATA/USB, PCIe/Fabrics)• General adoption of PCIe in all current ODM/OEM designs• Increasing interest in fabrics (see also CCIX announcement)

What is in an ARMv8 64-bit server class SoC?

13

64-bit ARMv8.x System-on-ChipsWhat is in an ARMv8 64-bit server class SoC? (Generic example)

DD

R DD

R

core corecore core core

core

core

core core core

14


DD

R DD

R


core

core

core core core

UART SATA R/XGMII PCIe

15


DD

R DD

R


core

core

core core core

UART SATA R/XGMII PCIe

Crypto Zlib Regex Switch

16

64-bit ARMv8.x server design flow

• Design goals determined (PPA – Power/Performance/Area)• Licensed vs custom IP development (Cortex-AXX/Custom)

• Core and system emulation (development of drivers)• System simulation (pre-silicon validation of design)• Tape-out of design (fabrication of silicon devices)• Post-silicon validation Respin/feature selection→

What is the design and development flow?

17


• Cores are becoming higher performance over time• Target is increasingly top shelf (“Xeon class”)• Fully Out-of-Order, super scalar, but also very efficient

• Adoption of industry standard bus architectures• PCIe-attached off-chip I/O rather than all integrated• Drop-in design alternative, supports ODM/OEM flexibility

Trends in ARMv8 SoC design/implementation...

18


• Clear differentiation between server and mobile class• Server designs target a common platform and provide

abstraction through management engines and firmware• Growth in what it means to be an ARMv8.x “server”

• IoT gateways, storage appliances, and telco infrastructure are moving toward commodity server in broader industry

Trends in ARMv8 SoC design/implementation...

19

64-bit ARMv8.x System Platforms

• Server Base Boot Requirements lays the “ground rules”• All systems boot using UEFI for platform firmware

• Abstraction, standardized boot, and (capsule) updates• Runtime services for common features (time, reboot, etc.)

• All systems use ACPI for hardware device enumeration• Abstracted generic platform with vendor managed hooks• Runtime method calls hide platform “glue” from OS

Standards-based firmware and Operating Systems

20

Swimming upstream...

21

Upstreaming (kernel) status...

• Upstream Linux has supported ARMv8.x for many years• Support for Enterprise class hardware initially lagged behind

• Industry lead specifications met with some resistance• ARM industry strength in diversity is also a weakness

• Enterprise use requires abstracted software platforms• Ship an OS today and also support future hardware

Improving over time but some more work to be done

22

Upstreaming (kernel) status...

• Embedded mentality leaked through into server early on• Early systems featured custom kernels with patches• Other distro vendors operate a “pay to play” model

• Enterprise use requires abstracted software platforms• Support for UEFI/ACPI platforms is now upstream• One kernel can now support most next gen systems

Improving over time but some more work to be done

23

Broader software ecosystem

• A wide range of Operating Systems now support ARMv8.x• e.g. CentOS, Debian, Fedora, OpenSUSE, RHELSA, Ubuntu…

• Software support includes typical dependencies, such as:• Kernel, Toolchains (GCC) and libraries (GLIBC)• Languages (OpenJDK), GoLang, Python, etc...• KVM and Xen virtualization

Operating Systems and higher level software components

24

Software defined everything on ARM

• Software Defined Infrastructure• Containerization (Docker)• OpenStack and OPNFV (DPDK and ARM VNFs)

• Software Defined Storage• Ceph clusters built on ARM servers• ARM powered disks running Ceph

Opportunities for ARM to shine in newly emerging markets

25

Linaro Enterprise Group (LEG)

• Linaro Enterprise Group (LEG) was formed in 2012• Initial focus on enabling 64-bit platform standards

• Provide reference software stacks• Base platform (OpenPlatformPkg)• Software Defined Infrastructure (OpenStack)• BigData and other examples

Developer resources and collaboration with Linaro – http://www.linaro.org/

26

Linaro Enterprise Group (LEG)

• Linaro Developer Cloud• Accelerating developer access to port their software• Applying a “dogfood” approach to testing SDI (OpenStack)

• Linaro Connect (Las Vegas)• Twice annual events are key collaboration opportunities

Developer resources and collaboration with Linaro – http://www.linaro.org/

27

MARCH 2011Red Hat ARMteam formed

2011 2012 2013 2014 2015 2016

Red Hat ARM Program Milestones5 years of investment in building a standardized 64-bit ARM server ecosystem

28


2011 2012 2013 2014 2015 2016

JAN 2012OpenJDK portfor AArch64


29


2011 2012 2013 2014 2015 2016


NOV 2012Red Hat co-foundedLinaro Enterprise Group


30


2011 2012 2013 2014 2015 2016



JAN 2014SBSA specificationreleased by ARM


31


2011 2012 2013 2014 2015 2016




JULY 2014Red Hat launches ARMPartner Early Access Program


32


2011 2012 2013 2014 2015 2016




JULY 2014Red Hat launches ARMPartner Early Access Program

JUNE 2015Red Hat announces RHEL Serverfor ARM Development Preview


33


2011 2012 2013 2014 2015 2016




JULY 2014Red Hat launches ARM Partner Early Access Program


JUNE 2015Red Hat announces RHEL Serverfor ARM Development Preview

JUNE 2016Red Hat demonstratesvirtualization POC

35

Red Hat Enterprise Linux Server for ARM

• RHELSA is a port of RHEL to the 64-bit ARMv8 Architecture• Currently shipping RHELSA7.2 (Development Preview)

• Supports SBSA/SBBR compliant ARM server SoCs• Platforms up to 4,096 cores, several TB RAM• Single kernel image with Enterprise features

• Virtual machine platform supports SBSA guests

A port of RHEL to the “AArch64” (ARMv8.X) Architecture, based upon RHEL7.x

plus.google.com/+RedHat

youtube.com/user/RedHatVideos

facebook.com/redhatinc

twitter.com/RedHatNewslinkedin.com/company/red-hat

THANK YOU

Enterprise class 64-bit ARM server update LinuxCon …...Enterprise class 64-bit ARM server update...

Documents

Transcript of Enterprise class 64-bit ARM server update LinuxCon …...Enterprise class 64-bit ARM server update...