The rCUDA Technology: Improvements Towards a Production ...€¦ · HPC Advisory Council Swiss...
Transcript of The rCUDA Technology: Improvements Towards a Production ...€¦ · HPC Advisory Council Swiss...
The rCUDA Technology:
Improvements Towards a
Production Ready Software
Federico SillaUniversitat Politècnica de València
Spain
HPC Advisory Council Swiss Conference 2017 2/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 3/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 4/30
rCUDA… CUDA… they sound similar
HPC Advisory Council Swiss Conference 2017 5/30
rCUDA… remote CUDA
rCUDA is a software technology that enables a more flexible use of GPUs
rCUDA allows a new envision of a GPU deployment:
Physical
configuration
Logicalconnections
Logical
configuration
Interconnection Network
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node nnode 2 node 3node 1
RAM RAM RAM RAM
node nnode 2 node 3node 1
Interconnection Network
Network
CPU
CPU RAM
RAM
Network
CPU
CPU RAM
RAM
Network
CPU
CPU RAM
RAM
Network
PC
Ie
PC
Ie
PC
Ie
PC
Ie
CPU
CPU RAM
RAM
GPU RAM GPU RAM GPU RAM GPU RAM
HPC Advisory Council Swiss Conference 2017 6/30
No GPU
rCUDA is a development by Universitat Politècnica de València, Spain
Basics of rCUDA
HPC Advisory Council Swiss Conference 2017 7/30
Basics of rCUDA
rCUDA is a development by Universitat Politècnica de València, Spain
HPC Advisory Council Swiss Conference 2017 8/30
Basics of rCUDA
rCUDA is a development by Universitat Politècnica de València, Spain
HPC Advisory Council Swiss Conference 2017 9/30
Performance of rCUDA
“Ideas Are Easy, Implementation Is Hard”
Guy Kawasaki, marketing specialist and
Silicon Valley venture capitalist
HPC Advisory Council Swiss Conference 2017 10/30
Performance of rCUDA
CPU to GPU
GPU to CPU
Higher
is better
HPC Advisory Council Swiss Conference 2017 11/30
Performance of rCUDA
CPU to GPU
GPU to CPU
Higher
is better
HPC Advisory Council Swiss Conference 2017 12/30
Performance of rCUDA
CPU to GPU
GPU to CPU
Higher
is better
HPC Advisory Council Swiss Conference 2017 13/30
Performance of applications using rCUDA
• K20 GPU and FDR InfiniBand
• K40 GPU and EDR InfiniBand
Lower
is better
HPC Advisory Council Swiss Conference 2017 14/30
Performance of applications using rCUDA
EDR InfiniBand and P100 GPU
CUDA-MEME
BarraCUDA
Lower
is better
Lower
is better
HPC Advisory Council Swiss Conference 2017 15/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 16/30
Why is P2P copy support needed?
rCUDA
model
CUDA
model
rCUDA scenario 1
rCUDA scenario 2
rCUDA must
provide the same
semantics as CUDA
HPC Advisory Council Swiss Conference 2017 17/30
Performance of P2P copies
Higher
is better
“Ideas Are Easy, Implementation Is Hard” Guy Kawasaki, marketing specialist and
Silicon Valley venture capitalist
HPC Advisory Council Swiss Conference 2017 18/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 19/30
Server consolidation
1
1
37
13
14
14
HPC Advisory Council Swiss Conference 2017 20/30
GPU-job migration within rCUDA
• rCUDA provides support for migrating jobs from one GPU in the cluster to another GPU located at the same or different node
• Only the GPU part of the application is migrated. The CPU part is not moved around
• Migration is transparent to applications, which are not aware that their GPU data and kernels have moved from one GPU to another
• When several jobs are sharing a given GPU, it is possible to migrate each of them independently to different destination GPUs
HPC Advisory Council Swiss Conference 2017 21/30
Example of migration performance
The GPU-Blast application is migrated up to 5 times among K40 GPUs
• The aggregated volume of GPU data is 1300 MB (consisting of 9 memory regions)
The “Reference” line is the execution time of the application when using CUDA with a local GPU and without any migration
Lower
is better
HPC Advisory Council Swiss Conference 2017 22/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 23/30
Deep Learning with CUDA
work in progress!!
Caffe
HPC Advisory Council Swiss Conference 2017 24/30
Deep Learning with rCUDA
very preliminary
results
Caffe
HPC Advisory Council Swiss Conference 2017 25/30
Outline
1. The rCUDA technology
2. P2P copies between GPUs
3. GPU job migration within rCUDA
4. Support for Deep Learning
5. Making rCUDA ready to industry
HPC Advisory Council Swiss Conference 2017 26/30
RoCE
CPU to GPU
GPU to CPU
HPC Advisory Council Swiss Conference 2017 27/30
Scheduling the shared use of GPUs
Interconnection Network
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node n
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 1
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 2
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 3
RAM RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 4
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 5
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 6
RAM
Network
GPU
PC
Ie
CPU
CPU RAM
RAM
node 7
RAM
Logicalconnections
Interconnection Network
Network
CPU
CPU RAM
RAM
node n
Network
CPU
CPU RAM
RAM
node 1
Network
CPU
CPU RAM
RAM
node 2
Network
CPU
CPU RAM
RAM
node 3
Network
CPU
CPU RAM
RAM
node 4
Network
CPU
CPU RAM
RAM
node 5
Network
CPU
CPU RAM
RAM
node 6
Network
PC
Ie
PC
Ie
PC
Ie
PC
Ie
PC
Ie
PC
Ie
PC
Ie
PC
Ie
CPU
CPU RAM
RAM
node 7
GPU RAM GPU RAM GPU RAM GPU RAMGPU RAM GPU RAM GPU RAM
Which GPU should I use?
HPC Advisory Council Swiss Conference 2017 28/30
Get a free copy of rCUDA at
http://www.rcuda.net
@rcuda_
More than 800 requests world wide
rCUDA is a development by Technical University of Valencia
HPC Advisory Council Swiss Conference 2017 29/30
Get a free copy of rCUDA at
http://www.rcuda.net
@rcuda_
More than 800 requests world wide
Jaime Sierra Pablo Higueras
rCUDA is a development by Technical University of Valencia
Carlos Reaño Javier Prades Tony Díaz
HPC Advisory Council Swiss Conference 2017 30/30
Thanks!
Questions?
rCUDA is a development by Technical University of Valencia