The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for...
Transcript of The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for...
![Page 1: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/1.jpg)
SC13 GPU Technology Theater
The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Peter Messmer, HPC Devtech
![Page 2: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/2.jpg)
Application
Computational Scientist’s Perspective
![Page 3: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/3.jpg)
Application
Computational Scientist’s Perspective
![Page 4: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/4.jpg)
Application
Computational Scientist’s Perspective
Hardware
![Page 5: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/5.jpg)
Application
Computational Scientist’s Perspective
Hardware
![Page 6: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/6.jpg)
Hardware
Hardware Designer’s Perspective
![Page 7: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/7.jpg)
Hardware
Hardware Designer’s Perspective
![Page 8: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/8.jpg)
Hardware
Application
Hardware Designer’s Perspective
![Page 9: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/9.jpg)
Hardware
Application
Hardware Designer’s Perspective
![Page 10: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/10.jpg)
Mismatch of Application and Hardware
![Page 11: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/11.jpg)
Hardware
Application
Co-Design Process
Application
![Page 12: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/12.jpg)
NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich
Foster collaboration between scientific user
community and NVIDIA
Get input from real-world hybrid applications
Support community in design decisions
Opened on November 1, 2013
Located on ETH Zurich campus
www.nvidiacodesignlab.ethz.ch
![Page 13: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/13.jpg)
Why ETH Zurich?
Leadership in Hybrid Computing in Europe
Parent organization of CSCS
Large investment in hybrid software
HP2C
PASC
Localized application expertise
![Page 14: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/14.jpg)
From HP2C to PASC
HP2C (2009 – 2013) Focus: Applications
PASC (2013 – 2016) Focus: Domain Networks
Material Science
Life Science Solid Earth Weather
Fluid Dynamics Cosmology
![Page 15: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/15.jpg)
Another reason for ETH Zurich..
Piz Daint at CSCS, 5272 K20X GPUs
#6 in Top500, #1 in Europe
![Page 16: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/16.jpg)
NVIDIA Co-Design Lab Near Term Activities
WP 1 Application Analysis and Optimization
WP 2 Future Architecture
Analysis
WP 3 Application
Visualization Environment
WP 4 Publication,
Outreach and Training
![Page 17: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/17.jpg)
Example: CP2K
Material science application
Innovative algorithm: O(N3) -> O(N)
New possibilities
Core algorithm: sparse matrix multiply
Small block matrices
Requires efficient tiny matrix-matrix product
Different from batched dgemm
![Page 18: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/18.jpg)
CP2K: Optimization of small batched DGEMM
Co-Design aspects
Input to application design
Input for future hardware
Possibly feed back into cuBLAS
batched 23x23x23
![Page 19: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/19.jpg)
COSMO: Study on Tri-Diagonal Solvers
Small tri-diagonal solves for vertical
coupling
Optimal algorithm unclear
Parallel Cyclic Reduction? Thomas?
How does hardware balance affect
performance?
![Page 20: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/20.jpg)
Outreach: Workshop Dec. 11, 2013
Presentations by existing and new
GPU users
Poll GPU activity at ETH/Zurich
www.nvidiacodesignlab.ethz.ch/news
![Page 21: The NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH …€¦ · NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich Foster collaboration between scientific](https://reader034.fdocuments.in/reader034/viewer/2022042915/5f5222fbdb38ab4f2066b633/html5/thumbnails/21.jpg)
Summary
NVIDIA Co-Design Center for Hybrid Multicore Computing at ETH
Zurich
www.nvidiacodesignlab.ethz.ch
Foster collaboration between NVIDIA and domain scientists
Provide input from real-world applications
Perform algorithm research for future generation systems
Open to everyone
Workshop on December 11, 2013