The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for...

21
SC13 GPU Technology Theater The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Peter Messmer, HPC Devtech

Transcript of The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for...

Page 1: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

SC13 GPU Technology Theater

The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Peter Messmer, HPC Devtech

Page 2: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Application

Computational Scientist’s Perspective

Page 3: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Application

Computational Scientist’s Perspective

Page 4: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Application

Computational Scientist’s Perspective

Hardware

Page 5: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Application

Computational Scientist’s Perspective

Hardware

Page 6: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Hardware

Hardware Designer’s Perspective

Page 7: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Hardware

Hardware Designer’s Perspective

Page 8: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Hardware

Application

Hardware Designer’s Perspective

Page 9: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Hardware

Application

Hardware Designer’s Perspective

Page 10: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Mismatch of Application and Hardware

Page 11: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Hardware

Application

Co-Design Process

Application

Page 12: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich

Foster collaboration between scientific user

community and NVIDIA

Get input from real-world hybrid applications

Support community in design decisions

Opened on November 1, 2013

Located on ETH Zurich campus

www.nvidiacodesignlab.ethz.ch

Page 13: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Why ETH Zurich?

Leadership in Hybrid Computing in Europe

Parent organization of CSCS

Large investment in hybrid software

HP2C

PASC

Localized application expertise

Page 14: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

From HP2C to PASC

HP2C (2009 – 2013) Focus: Applications

PASC (2013 – 2016) Focus: Domain Networks

Material Science

Life Science Solid Earth Weather

Fluid Dynamics Cosmology

Page 15: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Another reason for ETH Zurich..

Piz Daint at CSCS, 5272 K20X GPUs

#6 in Top500, #1 in Europe

Page 16: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

NVIDIA Co-Design Lab Near Term Activities

WP 1 Application Analysis and Optimization

WP 2 Future Architecture

Analysis

WP 3 Application

Visualization Environment

WP 4 Publication,

Outreach and Training

Page 17: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Example: CP2K

Material science application

Innovative algorithm: O(N3) -> O(N)

New possibilities

Core algorithm: sparse matrix multiply

Small block matrices

Requires efficient tiny matrix-matrix product

Different from batched dgemm

Page 18: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

CP2K: Optimization of small batched DGEMM

Co-Design aspects

Input to application design

Input for future hardware

Possibly feed back into cuBLAS

batched 23x23x23

Page 19: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

COSMO: Study on Tri-Diagonal Solvers

Small tri-diagonal solves for vertical

coupling

Optimal algorithm unclear

Parallel Cyclic Reduction? Thomas?

How does hardware balance affect

performance?

Page 20: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Outreach: Workshop Dec. 11, 2013

Presentations by existing and new

GPU users

Poll GPU activity at ETH/Zurich

www.nvidiacodesignlab.ethz.ch/news

Page 21: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific

Summary

NVIDIA Co-Design Center for Hybrid Multicore Computing at ETH

Zurich

www.nvidiacodesignlab.ethz.ch

Foster collaboration between NVIDIA and domain scientists

Provide input from real-world applications

Perform algorithm research for future generation systems

Open to everyone

Workshop on December 11, 2013