Integrating GPU Support for OpenMP Offloading...

Post on 21-Jun-2020

9 views 0 download

Preview:

Click to see full reader
Report this document
SHARE

Transcript of Integrating GPU Support for OpenMP Offloading...

Carlo Bertolli, Samuel F. Antao, Gheorghe-Teodor Bercea, Arpith C. Jacob, Alexandre E. Eichenberger, Tong Chen, Zehra Sura, Hyojin Sung, Georgios Rokos, David Appelhans, Kevin O’Brien

IBM T.J. Watson Research Center

The Second Workshop on the LLVM Compiler Infrastructure in HPC 11.15.15

Integrating GPU Support for OpenMP Offloading Directives into Clang

C/C++ input file

CPU GPU

Preproc. Preproc.

Clang Clang

LLVM P8

LLVM NVPTX

2

On Chip Controller(OCC) Overview Todd Rosedahl, Chief Engineer

IBM/POWER Firmware Development

Join the conversation at #OpenPOWERSummit 1

#OpenPOWERSummit libtarget-nvptx

LD

PTXAS

NVLINK

Host GPUFat binary

omp/lomp libtarget

P8 P8

K

K

K

K

K

K

M

M

M

M

M

M

MPCI Express

XLF

C/C++ input file

CPU GPU

Preproc. Preproc.

Clang Clang

LLVM P8

LLVM NVPTX

3

On Chip Controller(OCC) Overview Todd Rosedahl, Chief Engineer

IBM/POWER Firmware Development

Join the conversation at #OpenPOWERSummit 1

#OpenPOWERSummit libtarget-nvptx

LD

PTXAS

NVLINK

Host GPUFat binary

omp/lomp libtarget

OpenMP Implementation

in Clang CG

XLF

P8 P8

K

K

K

K

K

K

M

M

M

M

M

M

MPCI Express

Research Topics

• Implement OpenMP on GPU - Hard to do for programming constraints - Cannot re-use OpenMP on CPU (codegen +lib) - Performance guaranteed to be trash in many cases - What should be optimized?

• Integration into Open Source compiler - Cannot be disruptive to compiler design and implementation - Based on existing assumptions: OpenMP is implemented in Clang codegen - Gather community interest for this implementation to land

4

OpenMP Challenges for GPUs#pragma omp target teams { if (a[0]++ > 0) { #pragma omp parallel for for (int i = 0 ; i < n ; i++) { if (omp_get_thread_num () > 0) { #pragma omp simd for (int s = 0 ; s < 32 ; s++) { .. } } else { #pragma omp simd for (int s = 0 ; s < 4 ; s++) { .. } } } } else if(b[0]++ > 0) { #pragma omp parallel for for (int i = 0 ; i < n*2 ; i++) { .. } } }

5

Sequential within team: only team master executes this

#pragma omp target teams thread_limit(256) { if (a[0]++ > 0) { #pragma omp parallel for num_threads(128) for (int i = 0 ; i < n ; i++) { if (omp_get_thread_num () > 0) { #pragma omp simd for (int s = 0 ; s < 32 ; s++) { .. } } else { #pragma omp simd for (int s = 0 ; s < 4 ; s++) { .. } } } } else if(b[0]++ > 0) { #pragma omp parallel for for (int i = 0 ; i < n*2 ; i++) { .. } } }

6

Parallel threads: some threads are executing this in parallel

OpenMP Challenges for GPUs

#pragma omp target teams { if (a[0]++ > 0) { #pragma omp parallel for for (int i = 0 ; i < n ; i++) { if (omp_get_thread_num () > 0) { #pragma omp simd for (int s = 0 ; s < 32 ; s++) { .. } } else { #pragma omp simd for (int s = 0 ; s < 4 ; s++) { .. } } } } else if(b[0]++ > 0) { #pragma omp parallel for nowait for (int i = 0 ; i < n*2 ; i++) { .. } } }

7

Explicit and implicit divergence between threads

OpenMP Challenges for GPUs

#pragma omp target teams { if (a[0]++ > 0) { #pragma omp parallel for for (int i = 0 ; i < n ; i++) { if (omp_get_thread_num () > 0) { #pragma omp simd for (int s = 0 ; s < 32 ; s++) { .. } } else { #pragma omp simd for (int s = 0 ; s < 4 ; s++) { .. } } } } else if(b[0]++ > 0) { #pragma omp parallel for nowait for (int i = 0 ; i < n*2 ; i++) { .. } } }

8

No actual simd units on GPUs

OpenMP Challenges for GPUs

Control Loop Scheme

int tmp = 3; #pragma omp target teams \ thread_limit(5) \ map(tofrom:tmp,a[:n]) { tmp += 3; #pragma omp parallel for \ num_threads(4) for (int i = 0 ; i < n; i++) a[i] += tmp; tmp = -1; }

9

0 1 2 3 4

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1: if(tid > 4) break; // parallel reg. 1 if (tid == 0) nextState = SQ2; break; case SQ2: if(tid > 0) break; // sequential reg. 2 finished = true; break; }

__syncthreads(); }

Avoid dynamic parallelism and start all threads

Control Loop Scheme

int tmp = 3; #pragma omp target teams \ thread_limit(5) \ map(tofrom:tmp,a[:n]) { tmp += 3; #pragma omp parallel for \ num_threads(4) for (int i = 0 ; i < n; i++) a[i] += tmp; tmp = -1; }

10

0 1 2 3 4

0 1 2 3 4

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1: if(tid > 4) break; // parallel reg. 1 if (tid == 0) nextState = SQ2; break; case SQ2: if(tid > 0) break; // sequential reg. 2 finished = true; break; }

__syncthreads(); }

Control Loop Scheme

int tmp = 3; #pragma omp target teams \ thread_limit(5) \ map(tofrom:tmp,a[:n]) { tmp += 3; #pragma omp parallel for \ num_threads(4) for (int i = 0 ; i < n; i++) a[i] += tmp; tmp = -1; }

11

0 1 2 3 4

0 1 2 3 4

0 1 2 3 4

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1: if(tid > 3) break; // parallel reg. 1 if (tid == 0) nextState = SQ2; break; case SQ2: if(tid > 0) break; // sequential reg. 2 finished = true; break; }

__syncthreads(); }

Control Loop Scheme

int tmp = 3; #pragma omp target teams \ thread_limit(5) \ map(tofrom:tmp,a[:n]) { tmp += 3; #pragma omp parallel for \ num_threads(4) for (int i = 0 ; i < n; i++) a[i] += tmp; tmp = -1; }

12

0 1 2 3 4

0 1 2 3 4

0 1 2 3 4

0 1 2 3 4

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1: if(tid > 4) break; // parallel reg. 1 if (tid == 0) nextState = SQ2; break; case SQ2: if(tid > 0) break; // sequential reg. 2 finished = true; break; }

__syncthreads(); }

Control Loop & Clang

• Rules for modular integration • Do’s

• Extend OpenMP-related functions • Add new function calls • Add new runtime functions only for specific targets

• Don’ts • OpenMP target implementation influences every C/C++ construct • Add metadata and process OpenMP later when more convenient

13

Example: Codegen Control Loop for #target

void CGF::EmitOMPTargetDirective(..) { // control flow will lead to…

if (isTargetMode) CGM.getOpenMPRuntime().EnterTargetLoop();

// emit target region statements CGF.EmitStmt(CS->getCapturedStmt());

if (isTargetMode) CGM.getOpenMPRuntime().ExitTargetLoop(); }

14

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break;

} __syncthreads();

}

codegen

Example: Codegen Control Loop for #target

void CGF::EmitOMPTargetDirective(..) { // control flow will lead to…

if (isTargetMode) CGM.getOpenMPRuntime().EnterTargetLoop();

// emit target region statements CGF.EmitStmt(CS->getCapturedStmt());

if (isTargetMode) CGM.getOpenMPRuntime().ExitTargetLoop(); }

15

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break;

} __syncthreads();

}

setInsertPoint

Example: Codegen Control Loop for #target

void CGF::EmitOMPTargetDirective(..) { // control flow will lead to…

if (isTargetMode) CGM.getOpenMPRuntime().EnterTargetLoop();

// emit target region statements CGF.EmitStmt(CS->getCapturedStmt());

if (isTargetMode) CGM.getOpenMPRuntime().ExitTargetLoop(); }

16

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break;

} __syncthreads();

}

codegen until #parallel

Example: Codegen Control Loop for #parallel

void CGF::EmitOMPParallelDirective(..) { // control flow will lead to…

if (isTargetMode) CGM.getOpenMPRuntime().EnterParallel();

// emit parallel region statements CGF.EmitStmt(CS->getCapturedStmt());

if (isTargetMode) CGM.getOpenMPRuntime().ExitParallel(); }

17

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1:

if(tid > num_threads) break;

} __syncthreads();

}

Example: Codegen Control Loop for #target

void CGF::EmitOMPParallelDirective(..) { // control flow will lead to…

if (isTargetMode) CGM.getOpenMPRuntime().EnterParallel();

// emit parallel region statements CGF.EmitStmt(CS->getCapturedStmt());

if (isTargetMode) CGM.getOpenMPRuntime().ExitParallel(); }

18

nextState = SQ1; while(!finished) { switch(nextState) { case SQ1: if(tid > 0) break; // sequential reg. 1 nextState = PR1; break; case PR1:

if(tid > num_threads) break;

} __syncthreads();

}

setInsertPoint

Control Loop Overhead vs CUDA (1/2)

19

VectorAdd CUDA ControlLoop

#registers/thread 16 64

SharedMemory(bytes) 0 280

Occupancy 95.9% 26.6%

ExecutionTime(usec.) 1523.5 1988.5

#pragma omp target teams \ distribute parallel for \ schedule(static,1) for (i = 0 ; i < n ; i++) a[i] += b[i] + c[i];

for (int i = threadIdx.x + blockIdx.x * blockDim.x; i < n; i += blockDim.x * gridDim.x) a[i] += b[i] + c[i];

Nvidia Tesla K40m -maxregcount=64

Control Loop Overhead vs CUDA (2/2)

20

VectorMatrixAdd CUDA ControlLoop

#registers/thread 18 64

SharedMemory(bytes) 0 280

Occupancy 97.3% 49.5%

ExecutionTime(usec.) 70832.0 78333.0

#pragma omp target teams \ distribute parallel for \ schedule(static,1) for (i = 0 ; i < n ; i++) for (j = 0 ; j < n_loop ; j++) a[i] += b[i] + c[i*n_loop + j];

for (int i = threadIdx.x + blockIdx.x * blockDim.x; i < n; i += blockDim.x * gridDim.x) for (j = 0 ; j < n_loop ; j++) a[i] += b[i] + c[i*n_loop + j];

Nvidia Tesla K40m -maxregcount=64

Occupancy / Register Allocation

• Many reasons: - A while loop with a switch inside hits hard register allocation - In OpenMP 4.0 kernel parameters are passed as pointer to pointer ‣The kernel is allowed to do pointer arithmetic ‣This provokes an additional register for each parameter ‣Fixed by OpenMP 4.5: always pass pointer, what matters is the pointee address

- CUDA and LLVM backends for NVPTX are different: ‣CUDA uses libnvvm, which is shipped as a library ‣ LLVM uses the open source code in the trunk ‣Different optimization strategies

21

Optimizing “Good Cases”: LULESH

• Recurrent pragma patterns to be optimized • Some hints

- No OpenMP control flow divergence - No nested parallelism/pragmas - No hard stuff: locks, tasks, etc..

22

#pragma omp parallel for firstprivate(numNode) for( Index_t gnode=0 ; gnode<numNode ; ++gnode ) { Index_t count = domain.nodeElemCount(gnode) ; Index_t *cornerList = domain.nodeElemCornerList(gnode) ; Real_t fx_tmp = Real_t(0.0) ; Real_t fy_tmp = Real_t(0.0) ; Real_t fz_tmp = Real_t(0.0) ; for (Index_t i=0 ; i < count ; ++i) { Index_t elem = cornerList[i] ; fx_tmp += fx_elem[elem] ; fy_tmp += fy_elem[elem] ; fz_tmp += fz_elem[elem] ; } domain.fx(gnode) = fx_tmp ; domain.fy(gnode) = fy_tmp ; domain.fz(gnode) = fz_tmp ; }

Porting LULESH to OpenMP 4

#pragma omp parallel for firstprivate(numNode) for( Index_t gnode=0 ; gnode<numNode ; ++gnode ) { }

23

#pragma omp target teams distribute parallel for schedule(static,1) \ firstprivate(numNode) for( Index_t gnode=0 ; gnode<numNode ; ++gnode ) { }

OpenMP 4-ization

Implementation of Combined Construct

24

#pragma omp target teams distribute parallel for schedule(static,1) \ firstprivate(numNode) for( Index_t gnode=0 ; gnode<numNode ; ++gnode ) { } CUDA-style notation

for (int i = threadIdx.x + blockIdx.x * blockDim.x; i < n; i += blockDim.x * gridDim.x) {

g_node = i;

// codegen loop body }

Compiler: • Detect pragma combination • Prove absence of nested pragmas

Combined Directive - Vector Add

25

Vectoradd CUDA ControlLoop Combined

#registers/thread 16 64 21

SharedMemory(bytes) 0 280 0

Occupancy 95.9% 26.6% 96%

ExecutionTime(usec.) 1523.5 1988.5 1523.1

Control Loop Overhead vs CUDA (2/2)

26

Vector-MatrixAdd CUDA ControlLoop Combined

#registers/thread 18 64 30

SharedMemory(bytes) 0 280 0

Occupancy 97.3% 49.5% 97.7%

ExecutionTime(usec.) 70832.0 78333.0 70456.0

27

MeshSize ApplyAccelBCForNode CalcMonotonicQRegionForElems

CUDA ControlLoop Combined CUDA ControlLoop Combined#registers 6 64 22 32 64 64SharedMemory 0 280 0 0 280 0

123Occupancy 5.5% 6.7% 43.8% 5.9% 6.6% 5.6%ExecutionTime(μsec) 5.184 20.928 5.024 11.169 62.751 15.775

303Occupancy 6.1% 3.3% 14.7% 70.6% 26.5% 43.3%ExecutionTime(μsec) 5.568 22.912 4.96 29.184 178.18 67.296

1003Occupancy 27.8% 13.2% 33.8% 93.0% 26.4% 47.3%ExecutionTime(μsec) 6.72 50.944 14.976 1287.4 4833.2 2563.4

LULESH Kernels

28

MeshSize ApplyAccelBCForNode CalcMonotonicQRegionForElems

CUDA ControlLoop Combined CUDA ControlLoop Combined#registers 6 64 22 32 64 64SharedMemory 0 280 0 0 280 0

123Occupancy 5.5% 6.7% 43.8% 5.9% 6.6% 5.6%ExecutionTime(μsec) 5.184 20.928 5.024 11.169 62.751 15.775

303Occupancy 6.1% 3.3% 14.7% 70.6% 26.5% 43.3%ExecutionTime(μsec) 5.568 22.912 4.96 29.184 178.18 67.296

1003Occupancy 27.8% 13.2% 33.8% 93.0% 26.4% 47.3%ExecutionTime(μsec) 6.72 50.944 14.976 1287.4 4833.2 2563.4

LULESH Kernels

29

MeshSize ApplyAccelBCForNode CalcMonotonicQRegionForElems

CUDA ControlLoop Combined CUDA ControlLoop Combined#registers 6 64 22 32 64 64SharedMemory 0 280 0 0 280 0

123Occupancy 5.5% 6.7% 43.8% 5.9% 6.6% 5.6%ExecutionTime(μsec) 5.184 20.928 5.024 11.169 62.751 15.775

303Occupancy 6.1% 3.3% 14.7% 70.6% 26.5% 43.3%ExecutionTime(μsec) 5.568 22.912 4.96 29.184 178.18 67.296

1003Occupancy 27.8% 13.2% 33.8% 93.0% 26.4% 47.3%ExecutionTime(μsec) 6.72 50.944 14.976 1287.4 4833.2 2563.4

LULESH Kernels

30

MeshSize ApplyAccelBCForNode CalcMonotonicQRegionForElems

CUDA ControlLoop Combined CUDA ControlLoop Combined#registers 6 64 22 32 64 64SharedMemory 0 280 0 0 280 0

123Occupancy 5.5% 6.7% 43.8% 5.9% 6.6% 5.6%ExecutionTime(μsec) 5.184 20.928 5.024 11.169 62.751 15.775

303Occupancy 6.1% 3.3% 14.7% 70.6% 26.5% 43.3%ExecutionTime(μsec) 5.568 22.912 4.96 29.184 178.18 67.296

1003Occupancy 27.8% 13.2% 33.8% 93.0% 26.4% 47.3%ExecutionTime(μsec) 6.72 50.944 14.976 1287.4 4833.2 2563.4

LULESH Kernels

31

MeshSize ApplyAccelBCForNode CalcMonotonicQRegionForElems

CUDA ControlLoop Combined CUDA ControlLoop Combined#registers 6 64 22 32 64 64SharedMemory 0 280 0 0 280 0

123Occupancy 5.5% 6.7% 43.8% 5.9% 6.6% 5.6%ExecutionTime(μsec) 5.184 20.928 5.024 11.169 62.751 15.775

303Occupancy 6.1% 3.3% 14.7% 70.6% 26.5% 43.3%ExecutionTime(μsec) 5.568 22.912 4.96 29.184 178.18 67.296

1003Occupancy 27.8% 13.2% 33.8% 93.0% 26.4% 47.3%ExecutionTime(μsec) 6.72 50.944 14.976 1287.4 4833.2 2563.4

LULESH Kernels

Conclusion

• Generality of OpenMP has a large performance cost • even in simpler cases (no tasks, no locks, etc..)

• Optimized schemes are possible • More schemes in the near future • Interesting cases require control-flow analysis in Clang

• Optimize register allocation for control loop • Better low level optimizations in NVPTX • Optimize control loop scheme

32

Top related

ANDHRAPRADESH POWER GENERATION ...Soudha, Gunadala, Vijayawada-520004, Andhra Pradesh. 19. Contact Details Phone: 0866 – 2526802/305/310 Email: se-hpc2@apgenco.gov.in OR hydelpjt@gmail.com
 Documents
EasySVM: A Visual Analysis Approach for Open-Box …predictive-workshop.github.io/papers/vpa2014_9.pdf · Online Submission ID: EasySVM: A Visual Analysis Approach for Open-Box Support
 Documents
Intro to - University at Buffalo€¦ · Intro to 0. Setup on hpc2.org - Register on hpc2.org - Intro to Rstudio - Set up with Github - File transfer: WebDAV
 Documents
Using Docker in High Performance Computing in OpenPOWER Environment · 2019-03-01 · IBM Platform Computing Join the conversation at #OpenPOWERSummit 1 #OpenPOWERSummit . Your logo
 Documents
Proteus a Custom Platform for Hybrid Memory Research€¦ · for Hybrid Memory Research Kenneth Wright, Sr Director Rambus Labs Join the Conversation #OpenPOWERSummit. 2 •The Research:
 Documents
Foundation Technical Agenda · 2019-03-01 · Foundation Technical Agenda Jeff Brown, TSC Chair IBM/OpenPOWER Foundation #OpenPOWERSummit . Technical Agenda Open Specifications •Available
 Documents
lld-workshop.github.io · 2017-12-14lld-workshop.github.io
 Documents
Interference with the Expression of a Novel Human Polycomb ......Interference with the Expression of a Novel Human Polycomb Protein, hPc2, Results in Cellular Transformation and Apoptosis
 Documents
CUDAProgramming,and,Performance,Tuning,with, IBMXL ......Revolu’onizing,the, Datacenter JointheConversaon, #OpenPOWERSummit, CUDAProgramming,and,Performance,Tuning,with, IBMXL,Compilers,on,POWER8,Systems,
 Documents
Machine Learning Lifecycle - boss-workshop.github.io
 Documents
HPC2 and ubinuclein define a novel family of histone ...
 Documents
Visualizing Time Series Predictabilitypredictive-workshop.github.io/papers/vpa2014_13.pdf · Visualizing Time Series Predictability Cagatay Turkay, Member, IEEE Abstract— Predicting
 Documents
O LENICK HPC2 2015 - scientificspectator.com · the amount of sweat that is secreted to the skin surface. Bathing and washing will remove the antiperspirant gel. Re-application of
 Documents
One-click Hadoop Cluster Deployment on OpenPOWER Systems · One-click Hadoop Cluster Deployment on OpenPOWER Systems Pradeep K Surisetty IBM #OpenPOWERSummit
 Documents
Multimodal AI - alvr-workshop.github.io · Video Downstream Tasks Video QA Video-and-Language Inference Video Captioning Video Moment Retrieval Image Downstream Tasks VQA VCR NLVR2
 Documents
Integrating Predictive Visualization with the Epidemic ...predictive-workshop.github.io/papers/vpa2014_16.pdf · we are integrating predictive analytics into our EpiSimS analysis
 Documents
OpenPOWER and the Roadmap Aheadopenpowerfoundation.org/.../uploads/2016/04/5_Brad-McCredie.IBM_.pdf · Revolutionizing the Datacenter Join the Conversation #OpenPOWERSummit OpenPOWER
 Documents
LLVM-based Communication Optimizations for PGAS Programsllvm-hpc2-workshop.github.io/slides/Hayashi.pdf · LLVM-based Communication Optimizations for PGAS Programs Akihiro Hayashi
 Documents
Enabling Coherent FPGA Acceleration Allan Cantle, President & Founder Nallatech Join the conversation at #OpenPOWERSummit1 #OpenPOWERSummit.
 Documents
Porting and Optimizing Applications for AC922 servers using ......Speaker name, Title Company/Organization Name Join the Conversation #OpenPOWERSummit Porting and Optimizing Applications
 Documents

Copyright © 2022 FDOCUMENTS