ODCA Board Best Practice: High Performance Computing at BMW

HIGH PERFORMANCE COMPUTING @ BMW

Wolfgang Burke, BMW Group, 22.09.2014

AGENDA.

1 BMW Group – The company

2 HPC demands

3 HPC Cluster @ BMW

4 HPC Colocation

5 HPC Cloud

High Performance Computing @ BMW, 22.9.2014


BMW GROUP – THE COMPANY. KEY FIGURES.

The BMW Group is one of the ten largest industrial enterprises in Germany.

Brands: BMW, MINI, Rolls-Royce, BMW Motorrad.

Sales BM W Group 2013: 76,06 bn. Euro.

Deliveries of 1.963.798 cars in 2013.

Deliveries of 115.215 motorcycles in 2013.

29 manufacturing and assembly plants in 14 countries.

43 sales companies worldwide.

Presence in more than 140 countries.

110.351 employees at 31.12.2013.

Moses Lake

Spartanburg

Shenyang

Rayong Chennai

Kulim

Jakarta

Rosslyn Araquari

Manaus Production Plants

Assembly Plants

Engines / Components

BMW Group

Joint Venture

Contract Production

Regensburg Dingolfing

Berlin

Eisenach Leipzig

Wackersdorf

Landshut Munich

Kaliningrad

Born (2014)

Steyr

Graz

Hams Hall Oxford

Swindon Goodwood

San Luis Potosí (2019)

State 07/2014

Cairo

BMW GROUP PRODUCTION NETWORK WORLDWIDE. 29 LOCATIONS IN 14 COUNTRIES.


BMW GROUP – THE COMPANY. POSITIVE IMPULSES BY NEW MODELS.

Mini Hatch 5 doors

BMW i8

BMW X6

BMW 2 Series Active Tourer

BMW X4

BMW M4 Cabrio

AGENDA.


2 HPC demands

3 HPC Cluster @ BMW

4 HPC Colocation

5 HPC Cloud



HPC DEMANDS CHALLENGES BY INCREASING HPC USAGE

2007-07 2008-07 2009-07 2010-07 2011-07 2012-07 2013-07 2014-07

Cor

e ho

urs

Computation time demands per month


HPC DEMANDS INCREASING NUMBER OF APPLICATION AREAS USING SIMULATION

Aerodynamics

Stiffness, structural dynamics

Passive Safety

Operating strength

Powertrain

Production


HPC APPLICATIONS AERODYNAMIC SIMULATION


HPC APPLICATIONS FUEL INJECTION SIMULATION.


HPC APPLICATIONS. SIMULATION OF PRODUCTION PROCESSES.

Forming limit diagram of an experimental tool Simulation of the deep drawing process


HPC DEMANDS SIMULATION MODELS GETTING MORE DETAILED


APPLICATIONS ON HPC. PREDICTIVENESS IN MATERIAL BEHAVIOR.


HPC DEMANDS. SIMULATION REDUCES PROTOTYPES IN THE DESIGN PROCESS.

design stage 2 design stage 1 industrialisation 1 2 3

Reduced prototype driven design process (schematically):

“close to

production” cars

design stage 2 design stage 1 industrialisation 1 2 3

Old prototype driven design process (schematically):

prototypes

“design evaluation”

prototypes

“design validation”

“close to

production” cars

prototypes “design validation” (only in case of high innovations)

Benefits of the new process:

Straightforward design process.

Reduced number of prototypes

and cost savings.


HPC DEMANDS CRASH SIMULATION: FRONTAL IMPACT

Experimental model for method development


HPC DEMANDS. NUMBER OF LEGAL REQUIREMENTS INCREASE.

Heute

1990 1997 2005

FMVSS 208 old FMVSS 208 new Phase I

FMVSS 208 new Phase II

FMVSS 214 static FMVSS 214 static + dynamic

FMVSS 214 new MDB + Pole

FMVSS 301 old rear crash FMVSS 301 new Offset rear crash

FMVSS 216 roof crush test FMVSS 216 new

Occupant Containm.

ECE R95 side crash ECE R95 AE-MDB Advanced European

Movable Barrier

2003/102/EC

PDB or TRL Barrier

NHTSA Front to Front

BMW model

US

A

EC

E

US

A

EC

E

US

A

EC

E

US

A

US

A

EC

E

Frontal impact

Side impact

Rear impact Fuel tightness

Rollover

Pedestrian safety

Compatibility

ECE R12 steering column displacement

ECE R94 Offset frontal crash

2011


BMW product portfolio (status June 2014)

Increasing number of models, variants, facelifts

Diversity per model (e.g. left/right hand drive, motorization)

BMW market introduction 2014

HPC DEMANDS INCREASING NUMBER OF MODELS AND VARIANTS

AGENDA.


2 HPC demands

3 HPC Cluster @ BMW

4 HPC Colocation

5 HPC Cloud



HPC CLUSTER @ BMW HPC SERVICE WITH AN EXTENSIBLE RESOURCE POOL.

Shared, extensible resource pool

Optimized computational architecture & technologies

Transparent access

Jobflow & workload management control

Overall operations framework

(e.g. installation, administration, monitoring)


HPC CLUSTER @ BMW HPC SERVICE WITH AN EXTENSIBLE RESOURCE POOL.

Workload-Management

HPC Cluster

Engineers,

Virtual Dev.


HPC CLUSTER @ BMW IMPACT OF HIGHER VIRTUAL SIMULATION USAGE

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Floor space of HPC installation

Goals of the BMW HPC decisions

enable a scalable model for data center space requirements

deal with the increasing electricity prices

address carbon footprint caused by higher energy consumption

2008 2010 2012 2014 2016 2018 2020 2022 2024

Electricity price

2007-07 2008-07 2009-07 2010-07 2011-07 2012-07 2013-07 2014-07

Core-h per month

AGENDA.


2 HPC demands

3 HPC Cluster @ BMW

4 HPC Colocation

5 HPC Cloud



HPC CLUSTER @ BMW HPC CLUSTER DEVELOPMENT STEPS.

HPC private cloud

• On-demand resource reservation

• Transparent access (self-service portal)

• Standardized applications (services)

• Fully automated

• Zero downtime approach

• Cluster capacity scalability

• Maximum cluster utilization

• Cluster technological adaptability

• Cluster vendor independence

• Cluster best price purchasing



HPC public cloud

• Flexible capacity scalability

• Best price purchasing

• “HPC as a service”

Challenges:

• Remote location

• Unclear cloud model

(IaaS, PaaS, SaaS, …)

• Cloud integration

• Data security

• Organizational aspects

• …

HPC private cloud




• Fully automated









HPC public cloud




Challenges:

• Remote location




• Data security


• …

HPC Co-Location

• Space capacity scalability

• DC technology adaptability

• DC best price purchasing

Challenges:

• Remote location

• Data security


HPC private cloud




• Fully automated








HPC COLOCATION ICELAND FOR HPC COLOCATION

Data center operation with 100% CO2 emission free, renewable energy (geothermal and hydro)

Free air cooling nearly the whole year

Fixed low power cost for up to 20 years

High scalability in data center floor space

High bandwidth network connections available

Power Usage Effectiveness (PUE) < 1,2

1

1,2

1,43

1,8

0 0,5 1 1,5 2

Average PUE

BMW Data Center PUE

Iceland Colocation PUE

Best physical reachable PUE


HPC COLOCATION RELOCATION OF ELEMENTARY CELLS TO ICELAND.

Workload-Management

HPC Cluster

Engineers,

Virtual Dev.


HPC COLOCATION CHALLENGE LATENCY

Batch jobs: Only one big bulk (result) data transfer

Optimization of TCP stack parameters

Communication performance for bulk data transfer sufficient

Distance Munich - Keflavik: 2850 km

Latency BMW local 0,3 ms roundtrip

Latency BMW – Iceland 53 ms roundtrip

Transfer-Server

Cluster

Workload

Management

Storage system

Job submit

Result (bulk) data transfer

WAN connection

WAN connection


HPC COLOCATION TRANSFER TIMES AND OVERALL RUNTIMES FOR CRASH JOBS

Transfer time

EZ IS

0,73%

EZ IS

0,86%

EZ MUC EZ MUC

0,41% 0,41%

EZ MUC

0,41%

100% 98% 97% 97% 110% Problem size of crash job

HPC stages

local stages


HPC COLOCATION INCREMENTAL EXTENSION OF HPC APPLICATIONS IN ICELAND

including

Crash

HPC load share

successful

not successful

open

not tested

November 2012

Initial application tests in Iceland

HPC load share

released

not release

not successful

open

not tested

February 2013

Start of productive calculations in Iceland

HPC load share

released

not successful

open

Current situation

Only a few applications open or not released

Crash


HPC CLOUD COLOCATION ODCA CARBON FOOTPRINT AND ENERGY EFFICIENCY USAGE MODEL

Application of the ODCA Carbon Footprint and Energy Efficiency Usage Model

Execution of an ODCA Carbon Footprint and Energy Efficiency Usage Model Proof of Concept


1148 673 2,1 0

200

400

600

800

1000

1200

1400

Coal Fired Grid Natural Gas Grid Iceland Data Center

Carbon Usage Effectiveness (CUE)

HPC COLOCATION DATA CENTER ICELAND: CARBON POC

g/kWh

AGENDA.


2 HPC demands

3 HPC Cluster @ BMW

4 HPC Colocation

5 HPC Cloud




HPC public cloud




Challenges:

• Remote location




• Data security


• …

HPC Co-Location

• Space capacity scalability

• DC technology adaptability

• DC best price purchasing

Challenges:

• Remote location

• Data security


HPC private cloud




• Fully automated








HPC CLOUD MOVE ELEMENTARY CELLS IN A HPC CLOUD.

Workload-Management

HPC Cluster

Engineers,

Virtual Dev.

HPC colocation provider HPC on premise



Workload-Management

HPC Cluster

Engineers,

Virtual Dev.

Stepwise approach

Reference performance benchmarks

HPC colocation provider HPC cloud provider HPC on premise

Cloud platform



HW provisioning performed by cloud platform

Servers

File service (scratch)

High speed network (Infiniband) • Gateway-Nodes

• ~132 Compute-

Nodes:

o Bignodes

o Stdnodes

• High Speed

Cluster Interconnect

• Admin Interconnect

EZ-Scratch

Building Block: Elementarzelle

WAN connection

HW administration provided by cloud platform

On-demand provisioning of servers in a cell

Cloud platform



• Gateway-Nodes

• ~132 Compute-

Nodes:

o Bignodes

o Stdnodes

• High Speed



EZ-Scratch


WAN connection 1. Step: Bare metal provisioning

HW server

OS provisioning

HPC provisioning cloud service operation

Reference for performance benchmarks

Cloud platform



• Gateway-Nodes

• ~132 Compute-

Nodes:

o Bignodes

o Stdnodes

• High Speed



EZ-Scratch


WAN connection 1. Step: Bare metal provisioning

HW server

OS provisioning

HPC provisioning cloud service operation

2. Step: Virtual server provisioning

cloud service operation

HPC provisioning

Virtual server

provisioning

Reference for performance benchmarks



Seamless integration of HPC cloud into BMW HPC cluster

First test results on bare metal show comparable performance to current cluster

Basis for a HPC service specification

Open questions to be clarified


HPC CLOUD DEVELOPMENT OF A HPC CLOUD MODEL.

ODCA usage models might be helpful on different layers

• Data Security Security Workgroup

• Long Distance Migration Manageability & Services

Workgroup

• ODCA Interop Across Clouds Guide

• Compute Infrastructure as a Service Infrastructure Workingroup

THANK YOU VERY MUCH FOR YOUR ATTENTION

Imprint:

Published by:

BMW Group

Corporate and Governmental Affairs

80788 Munich

No reproduction, in whole or in part, without the express permission of Bayerische Motoren Werke Aktiengesellschaft, München ©2014 Bayerische Motoren Werke Aktiengesellschaft

ODCA Board Best Practice: High Performance Computing at BMW

Technology

Transcript of ODCA Board Best Practice: High Performance Computing at BMW