OpenPOWER Foundation Technical Overview - IBM · • Numerous disruptive forces are impacting these...

© 2014 IBM Corporation

LinuxCon Japan 2014

OpenPOWER Technical Overview

Jeff ScheelChief EngineerLinux on Power

May 21, 2014

[email protected]

mailto:[email protected]

https://plus.google.com/communities/100156952249293416679

http://twitter.com/@ibmpowerlinux

https://www.ibm.com/developerworks/mydeveloperworks/blogs/fe313521-2e95-46f2-817d-44a4f27eba32/tags/jscheel

© 2014 IBM Corporation2

Agenda

1. OpenPOWER Foundation Overview

2. OpenPOWER Hardware Technologies

3. OpenPOWER System Software Architecture


OpenPOWER Foundation Overview

4 © OpenPOWER Foundation 2014

Industry trends

• The number of companies designing & building servers is increasing

– Traditionally there have been few companies designing systems: HP, IBM, SUN, Dell, etc.

– Today there are many more: Google, Microsoft, Facebook, Rackspace, Huawei, Sugon, Inspur, etc.

– A fairly mature ecosystem including the Taiwanese ODMs is a key enabler of this trend

• Numerous disruptive forces are impacting these custom system designs and driving designers to consider new ways of innovating

– Ability to handle rapid growth in Big Data & Analytics based solutions– Choice and Innovation– CPU SOC integration drive need for chip development

• These trends create a need for a server targeted “chip-system-software” ecosystem

– IBM has technology and a software stack ready to meet these needs– IBM recognizes the need to work with partners to create this ecosystem– IBM recognizes the need for choice and options in processor sourcing


OpenPOWER Foundation Structure

OpenPOWER is an industry foundation based on the POWER architecture, enabling an Open community for development and opportunity for member differentiation and growth


The goal of the OpenPOWER Foundation is to create an open ecosystem, using the POWER Architecture to share expertise, investment, and

server-class intellectual property to serve the evolving needs of customers.

OpenPOWER drives industry innovation

– Opening the architecture to give the industry the ability to innovate across the full Hardware and Software stack

• Simplify system design with alternative architecture

• Includes SOC design, Bus Specifications, Reference Designs, FW OS and Open Source Hypervisor

• Little Endian Linux to ease the migration of software to POWER

– Driving an expansion of enterprise class Hardware and Software stack for the data center

– Building a complete ecosystem to provide customers with the flexibility to build servers best suited to the Power architecture


Building collaboration and innovation at all levels

Welcoming new members in all areas of the ecosystem100+ inquiries and numerous active dialogues underway

Boards/Systems

I/O, Storage, Acceleration

Chip/SOC

System/Software/Services


Proposed Work Groups and Projects

Work Group Projects Participants

System Software(Open Source)

•Linux LE•KVM•Firmware

–OpenPOWER FW interface

PublicPublicPublic

•POWER LE ABI Public

Application Software(Open Source)

•System Operating Environment– OpenPOWER Software ecosystem

enablement

Public

•Toolchain Public

Open Server Development Platform

•Power 8 Developer Board Member

•POWER 8 Reference Design Member

Hardware Architecture

•OpenPOWER profile of architecture – Power8 ISA Book 1, 2, 3

Member

•Coherent Accelerator Interface Architecture (CAIA)

Member

Compliance •Compliance Member


OpenPOWER Innovation Roadmap

Domain Project ParticipantsChip design ISA Standard Products/ SOCs IBM, Suzhou Power Corp

Systems CM/ODM/OEM models Tyan, Servergy, Inspur, ZTE, ChuangHe,IBM, Hitachi

Memory Processor - Memory Interface optimizations

Hynix, Micron, Samsung

I/O Network Optimization- RDMA Performance- tightly coupled NIC-processor

Storage Optimization

Mellanox

Fusion I/O

Acceleration Tightly coupled custom FPGA accelerators

Enterprise & HPC Workloads

Xilinx, Altera

NVIDIA

Software Cloud Optimized Firmware Stack & LE Linux

Google, Canonical , IBM


OpenPOWER System Software Overview

• Draft charter proposes working group to be “stewards, incubators, and drivers (when feasible) of system software enablement for OpenPOWER Foundation hardware”

• Open to any members of the foundation

• Key workgroup products include:– OpenPOWER Foundation web presence– Creation of new open source communities– Complete open source software stack from firmware to system management software– Documentation of key community interfaces including LE ABI, firmware specifications,

and virtualization APIs

• Jointly works with HW Architecture and Compliance workgroups to:

– Define and manage the POWER ISA– Ensure compliance to software interfaces– Implement/exploit new technology such as CAPI


OpenPOWER Hardware Technologies


POWER8 Processor

Caches • 512 KB SRAM L2 / core• 96 MB eDRAM shared L3• Up to 128 MB eDRAM L4

(off-chip)

Cores • 12 cores (SMT8)• 8 dispatch, 10 issue,

16 exec pipe• 2X internal data

flows/queues• Enhanced prefetching• 64K data cache,

32K instruction cache

Accelerators• Crypto & memory expansion• Transactional Memory • VMM assist • Data Move / VM Mobility Energy Management

• On-chip Power Management Micro-controller• Integrated Per-core VRM• Critical Path Monitors

Technology•22nm SOI, eDRAM, 15 ML 650mm2

Memory• Up to 230 GB/s

sustained bandwidth

Bus Interfaces• Durable open memory

attach interface• Integrated PCIe Gen3• SMP Interconnect• CAPI (Coherent

Accelerator Processor Interface)

ComputerWorld: To make the chip faster, IBM has turned to a more advanced manufacturing process, increased the clock speed and added more cache memory, but perhaps the biggest change heralded by the Power8 cannot be found in the specifications. After years of restricting Power processors to its servers, IBM is throwing open the gates and will be licensing Power8 to third-party chip and component makers. The Register: the Power8 is so clearly engineered for midrange and enterprise systems for running applications on a giant shared memory space, backed by lots of cores and threads. Power8 does not belong in a smartphone unless you want one the size of a shoebox that weighs 20 pounds. But it most certainly does belong in a badass server, and Power8 is by far one of the most elegant chips that Big Blue has ever created, based on the initial specs. PCWorld: With Power8, IBM has more than doubled the sustained memory bandwidth from the Power7 and Power7+, to 230 GB/s, as well as I/O speed, to 48 GB/s. Put another way, Watson’s ability to look up and respond to information has more than doubled as well.

Microprocessor report: Called Power8, the new chip delivers impressive numbers, doubling the performance of its already powerful predecessor, Power7+. Oracle currently leads in server-processor performance, but IBM’s new chip will crush those records. The Power8 specs are mind boggling.

Source: Hotchips presentation


POWER8 CAPI

CustomHardware

Application

POWER8

CAPP

Coherence Bus

PSL

FPGA or ASIC

Customizable HardwareApplication Accelerator • Specific system SW, middleware, or user application• Written to durable interface provided by PSL

POWER8

PCIe Gen 3Transport for encapsulated messages

Processor Service Layer (PSL)• Present robust, durable interfaces to applications• Offload complexity / content from CAPP

Virtual Addressing• Accelerator can work with same memory addresses that the

processors use• Pointers de-referenced same as the host application• Removes OS & device driver overhead

Hardware Managed Cache Coherence• Enables the accelerator to participate in “Locks” as a normal thread

Lowers Latency over IO communication model

Coherent Accelerator Processor Interface (CAPI)


Coherent Accelerator Processor Interface (CAPI) Overview

CAPP PCIe

POWER8 Processor

Typical I/O Model Flow

Flow with a Coherent Model

Shared Mem. Notify Accelerator

AccelerationShared Memory

Completion

DD CallCopy or PinSource Data

MMIO NotifyAccelerator

AccelerationPoll / Int

CompletionCopy or UnpinResult Data

Ret. From DDCompletion

FPGA

Fu

nctio

n n

Fu

nctio

n 0

Fu

nctio

n 1

Fu

nctio

n 2

CAPI

IBM Supplied POWER Service Layer


Example: Innovative “In-Memory” NoSQL/KVS Integrated Solution - via POWER8 CAPI-attached Flash

WWW

10Gb Uplink

POWER8 Server

Flash Array w/ up

to 40TB

Differentiated NoSQL(POWER8 + CAPI Flash)

Infrastructure Attributes- 192 threads in 4U Server drawer

- 40 TB of memory based Flash per 4U Drawer- Shared Memory & Cache for dynamic tuning

- Elimination of I/O and Network Overhead- Cluster solution in a box

5X Cost Reduction with

equivalent performance

WWW

500GB Cache Node500GB

Cache Node500GB Cache Node500GB

Cache Node500GB Cache Node500GB

Cache Node

Backup Node

Load Balancer

Today’s NoSQLin memory (x86)

10Gb Uplink

Infrastructure Requirements- Large Distributed (Scale out)

- Large Memory per node- Networking Bandwidth Needs

- Load Balancing

Power CAPI-attached Flash model for NoSQL offers dramatic (24:1) density advantage


Power Systems S822LPower Systems

S812L• 1-socket, 2U• Linux Only

• 2-socket, 2U• Linux Only

• 2-socket, 2U• All Operating Systems

Power Systems S822

Power Systems S814

• 1-socket, 4U• All Operating Systems

Power Systems S824

• 2-socket, 4U• All Operating

SystemsPower Systems S824L

• 2-socket, 4U• Linux Only• SOD

1 & 2 Sockets

New IBM Power Systems based on POWER8


Non-IBM POWER8 products

http://www.enterprisetech.com/2014/04/28/inside-google-tyan-power8-server-boards/

The Tyan reference (ATX) board, SP010, measures 12” by 9.6”➢ one single-chip module (SCM)➢ four DDR3 memory slots➢ four 6 Gb/sec SATA peripheral connectors➢ two USB 3.0 ports➢ two Gigabit Ethernet network interfaces➢ keyboard and video➢ intended for developers

The Google reference board➢ two single-chip module (SCM)➢ four modified SATA ports➢ Google use only

http://www.enterprisetech.com/2014/04/28/inside-google-tyan-power8-server-boards/


OpenPOWER System Software Architecture


OpenPOWER Proposed Ecosystem Enablement

XCATXCAT

CloudSoftware

OperatingSystem / KVM

Standard OperatingEnvironment

(System Mgmt)

So

ftw

are

Power Open Source Software Stack Components

ExistingOpen

Source Software

Communities

Firmware

Hardware

New OSS Community

OpenPOWERTechnology

OpenPOWERFirmware

CAPP

PC

Ie

POWER8

CAPI over PCIe

“Standard POWER Products” – 2014

Har

dw

a re

“Custom POWER SoC” – Future

Customizable

Framework to Integrate System IP on Chip

Industry IP License Model

Multiple Options to Design with POWER Technology Within OpenPOWER


KVM and OpenStack enable “defacto” APIs for integration

Power8 Platform

OpenPOWER FW

Qemu

VM1Linux

VM2Linux

LibvirtAPI & virsh CLI

Linux Kernel

Hos

t O

S

ConsoleShell CLI

Linux UserspaceOpenstack

compute nodecomponents

Openstackcontrollerservices

xCatChef

PuppetCustom scripts

ProjectKimchi

BrowserOr

Client

IPMIServiceProcessor

KVM

VM3Linux

Note: oVirt has been enabled in version 3.4 for Linux on Power.

Standard interfaces


Integrating POWER into a datacenter

H/W Management- Custom scripting- Heavily Automated- Open technologies

Platform Management- Custom management- Cloud based

In general– Bring base components in parity with KVM on x86 – “KVM should be KVM”– Co-exists peacefully with other end-points, whenever makes sense.– Enables hybrid clouds with common management environments

Manage

Provision

Deploy

Procure

xCatPuppetChefCustom Scripts

PXEIPMI

OpenStackCustom Scripts


OpenPOWER Enables Hybrid Cloud

• Linux – to provide commonality for:– Operating system management

– Operating system feature

– Application programming model

• Little Endian – to provide source code and data commonality

• KVM – to provide virtualization management and feature commonality

• Firmware interfaces – to provide platform management commonality

• Software written in interpreted languages (Javascript, PHP, Perl, Python, Ruby, Java, etc.)

– Generally, no work is required.

• Software written in compiled languages (C/C++, Fortran, etc.)

– Generally, this requires just a simple recompile for POWER.

• Rarely, dependencies on specific behaviors can require source code modification:

– Multi-threaded applications that don’t use standard synchronization models and depend upon specific memory ordering behavior (unusual)

– Applications that depend upon specific memory page sizes (rare)

OpenPOWER software commonality Migrating software to OpenPOWER


POWER8 supports little endian data access, like x86

● POWER8 processors support execution in both big endian (BE) and little endian mode (LE)

● Today's POWER operating systems run the processors in BE mode – Linux, AIX, and IBM i

● Most compiled open source software is designed (defacto) to run in little endian mode.

● Linux on Power has chosen to exploit little endian (LE) processor mode based on OpenPOWER partner feedback.

– Eases the migration of applications from Linux on x86.– Enables simple data migration from Linux on x86.– Simplifies data sharing (interoperability) with Linux on x86.– Improves Power I/O offerings with modern I/O adapters and

devices, e.g. GPUs.

● Creating LE OS for Linux on Power means creating a whole new operating system “platform” (ppc64le).

● LE distributions for Linux on Power does NOT mean x86 applications magically run: applications must still be compiled for Power.

BE and LE are simply different ways of ordering of how data is stored. Important in 1980.Not as important in 2013.Market has moved.


Wrap-up


Where to find more information? http://openpowerfoundation.org/

http://openpowerfoundation.org/


Q & A

Summary:

1.The OpenPOWER Foundation provides a framework for open innovation on hardware and software

2.The system software structure embraces many key open source technologies – KVM, OpenStack, and other

3.Opportunities abound for hardware innovation with the POWER8 processor as a foundation

OpenPOWER Foundation Technical Overview - IBM · • Numerous disruptive forces are impacting these...

Documents

Transcript of OpenPOWER Foundation Technical Overview - IBM · • Numerous disruptive forces are impacting these...