Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance...

14
Trish Damkroger Vice President, Data Center Group General Manager, Technical Computing Initiative

Transcript of Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance...

Page 1: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Trish DamkrogerVice President, Data Center GroupGeneral Manager, Technical Computing Initiative

Page 2: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Before We Begin - The coming flood of dataBy 2020…

The average internet user will generate

~1.5 GB of traffic per daySmart hospitals will be generating over

3,000 GB per day

A connected plane will be generating over

40,000 GB per dayA connected factory will be generating over

1,000,000 GB per day

Self driving cars will be generating over

4,000 GB Per day … Each

The world is changing – without Analytics and aI workloads we will drown in data!!!

Page 3: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Different Systems (Today)

FORTRAN / C++ ApplicationsMPI

High Performance

Java* ApplicationsHadoop*

Simple to Use

SLURMSupports large scale startup

YARN*More resilient of hardware failures

Lustre*Remote Storage

HDFS*, SPARK*Local Storage

Compute & Memory FocusedHigh Performance Components

Storage FocusedStandard Server Components

Server StorageSSDs

SwitchFabric

Infrastructure

ProgrammingModel

Resource Manager

File System

Hardware

Server StorageHDDs

SwitchEthernet

Infrastructure

3Daniel Reed and Jack Dongarra, Exascale Computing and Big Data in Communications of the ACM journal, July 2015 (Vol 58, No.7), and Intel analysisOther brands and names are the property of their respective owners.

Page 4: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Emerging Real-Time Workflows

Small Data + Small Compute

e.g. EDA

Big Data + Small Compute e.g. Search, Streaming, Data Preconditioning

Small Data +Big Compute

e.g. Mechanical Design, Multi-physics

Da

ta

Compute

4

Page 5: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

One System Architecture?

From Here … …TO?

5

Page 6: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

The Challenges of Moving to (HPC && Big Data)

Source: NERSC Workload Analysis on Hopper. K. Antypas et al. 2014

Many Codes on a System

Over 600 Different Applications on NERSC’s Hopper System

6

Page 7: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

The Challenges of Moving to (HPC && Big Data)

Source: NERSC Workload Analysis on Hopper. K. Antypas et al. 2014

Many Codes on a System

Over 600 Different Applications on NERSC’s Hopper System

Source: Intel for illustration purposes

Varied Resource Needs

Typical HPC Workloads

High Frequency

Trading

Numeric Weather

Simulation

Oil & Gas Seismic

Sy

ste

m c

ost

ba

lan

ce

Typical Big Data Workloads

Video Survey Traffic Monitor

Personal Digital Health

Sy

ste

m c

ost

ba

lan

ce

Processor Memory Interconnect Storage

7

Page 8: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Benefits of a Single System Architecture

Flexibility to manage workflows and maximize system utilization

234

1

Improved Total Cost of Ownership

Overcomes cost and complexity of moving data

Potential for a unified SW ecosystem

8

Page 9: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Realizing the Benefits Requires Coordinated Design

ComputeMany core

Multicore

Intel Graphics

Intel Xeon + FPGA

System Software StackSimplifies Use and Application Development

InterconnectConfigurable

Next-Generation Fabric

Memory/StorageConfigurable High BW Memory

Non-Volatile MemoryStorage Burst BufferLustre File System

9

Page 10: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Converged Architecture for HPC and Big Data

10

FORTRAN / C++ ApplicationsMPI

High Performance

Java* ApplicationsHadoop*

Simple to Use

Lustre* with Hadoop* AdapterRemote Storage

Compute & Big Data CapableScalable Performance Components

Server Storage(SSDs and Burst Buffers)

Intel®Omni-Path

Architecture

Infrastructure

Programming Model

Resource Manager

File System

Hardware

*Other names and brands may be claimed as the property of others

HPC & Big Data-Aware Resource Manager

Page 11: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Key Enabler: A Configurable Memory-Storage Hierarchy

11

Processor

Compute Node

I/O Node

Remote Storage

Compute

Today

Caches

Local Memory

SSD Storage

Parallel File System(Hard Drive Storage)

Hig

he

r B

an

dw

idth

. L

ow

er

La

ten

cy a

nd

Ca

pa

city

Some remote data moves onto I/O node

I/O Node storage moves to compute node

Local memory is now faster & in processor package

Compute

Future

Caches

Non-Volatile Memory

Burst Buffer Storage

Parallel File System (Hard Drive Storage)

In-Package High Bandwidth Memory*

*cache, memory or hybrid mode

Page 12: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

MachineLearning

Modeling & Simulation

High PerformanceData Analytics

Many workloads | One architectural framework

Small Clusters Through Supercomputers

Standards-Based Programmability

On-Premise and Cloud-Based

Intel® Scalable System Framework

Compute and Data-Centric

Page 13: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC

Legal Disclaimers

Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on

system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at [intel.com].

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark,

are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should

consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with

other products.

All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps.

Results have been estimated or simulated using internal Intel analysis or architecture simulation or modeling, and provided to you for informational purposes. Any differences in

your system hardware, software or configuration may affect your actual performance.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on

system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at https://www-

ssl.intel.com/content/www/us/en/high-performance-computing/path-to-aurora.html.

Intel, the Intel logo, Xeon, and Intel Xeon Phi are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries.

*Other names and brands may be claimed as the property of others.

© 2015 Intel Corporation

13

Page 14: Trish Damkroger Vice President, Data Center Group General … · 2017-09-01 · High Performance Java* Applications Hadoop* Simple to Use SLURM Supports large scale startup ... HPC