Parallel and Pipeline Programming. Super- scalar, pipelined with vector instruction support.
-
Upload
abigayle-audra-sims -
Category
Documents
-
view
221 -
download
0
Transcript of Parallel and Pipeline Programming. Super- scalar, pipelined with vector instruction support.
Parallel and Pipeline Programming
Super-scalar,
pipelined with vector instruction
support
Definitions
• Super-scalar - multiple integer or floating-point ALUs
• Pipeline - executes instructions in steps like an assembly line
• Stall - instruction execution state that delays a pipeline step– If an add takes 2 steps and there are two ALUs,
then 3 adds in a row could cause a stall
Memory Hierarchy
Access Times
Hierarchy Access Times
To Where CPU CyclesRegister <= 1L1d cache ~3L2 cache ~14L3 cache ~30Main Memory ~240Disk ~7,000,000
Disk Transfer Time
• Fujitsu MHS2060AT 60GB Laptop Hard Drive
• 4200RPM, 420 sectors/track, 512 B/sector
• 1 track per 1/4200M = 1/70S
• 1 track/(1/70S) x 420 sect/trk x 512 B/sect
• = 15.05mb/second transfer rate
Key to effective cache utilization is program locality
• Temporal locality refers to the reuse of the same address within relatively small time durations.
• Spatial locality refers to the use of data within relatively "close" storage locations.
Page Fault-Rate Curve from OS
Multiprocessor Caching
Sequential Consistency
R/W from each CPU
reach memory in
order executed
Strict Consistency• R/W are seen in same order by all processors• In hardware, is implemented by atomic
hardware instructionsIntel Compare-Exchange Semantics
if (EAX== DEST) {ZF = 1 DEST = SRC
} else { ZF = 0 EAX= DEST}
Processor Code Improvement
Gcc Optimization Options-falign-functions=n -falign-jumps=n -falign-labels=n -falign-loops=n -falign-loops-max-skip=n -falign-jumps-max-skip=n
-fbounds-check -fmudflap -fmudflapth -fmudflapir -fbranch-probabilities -fprofile-values -fvpt -fbranch-target-load-optimize
-fbranch-target-load-optimize2 -fbtr-bb-exclusive -fcaller-saves -fcprop-registers -fcreate-profile -fcse-follow-jumps
-fcse-skip-blocks -fcx-limited-range -fdata-sections -fdelayed-branch -fdelete-null-pointer-checks -fearly-inlining
-fexpensive-optimizations -ffast-math -ffloat-store -fforce-addr -ffunction-sections -fgcse -fgcse-lm -fgcse-sm -fgcse-las
-fgcse-after-reload -fcrossjumping -fif-conversion -fif-conversion2 -finline-functions -finline-functions-called-once
-finline-limit=n -fkeep-inline-functions -fkeep-static-consts -flocal-alloc (APPLE ONLY)-fmerge-constants
-fmerge-all-constants -fmodulo-sched -fno-branch-count-reg -fno-default-inline -fno-defer-pop -fmove-loop-invariants
-fno-function-cse -fno-guess-branch-probability -fno-inline -fno-math-errno -fno-peephole -fno-peephole2
-funsafe-math-optimizations -funsafe-loop-optimizations -ffinite-math-only -fno-toplevel-reorder
-fno-trapping-math -fno-zero-initialized-in-bss -mstackrealign -fomit-frame-pointer -foptimize-register-move
-foptimize-sibling-calls -fprefetch-loop-arrays -fprofile-generate -fprofile-use -fregmove -frename-registers -freorder-blocks
-freorder-blocks-and-partition -freorder-functions -frerun-cse-after-loop -frounding-math -frtl-abstract-sequences
-fschedule-insns -fschedule-insns2 -fno-sched-interblock -fno-sched-spec -fsched-spec-load
-fsched-spec-load-dangerous -fsched-stalled-insns=n -fsched-stalled-insns-dep=n
-fsched2-use-superblocks -fsched2-use-traces -fsee -freschedule-modulo-scheduled-loops
-fsection-anchors -fsignaling-nans -fsingle-precision-constant -fstack-protector -fstack-protector-all -fstrict-aliasing
-fstrict-overflow -ftracer -fthread-jumps -funroll-all-loops -funroll-loops -fpeel-loops -fsplit-ivs-in-unroller -funswitch-loops
-fvariable-expansion-in-unroller -ftree-pre -ftree-ccp -ftree-dce -ftree-loop-optimize
-ftree-loop-linear -ftree-loop-im -ftree-loop-ivcanon -fivopts -ftree-dominator-opts
-ftree-dse -ftree-copyrename -ftree-sink -ftree-ch -ftree-sra -ftree-ter -ftree-lrs -ftree-fre
-ftree-vectorize -ftree-vect-loop-version -ftree-salias -fuse-profile -fipa-pta -fweb -ftree-copy-prop
-ftree-store-ccp -ftree-store-copy-prop -fwhole-program --param name=value
-O -O0 -O1 -O2 -O3 -Os -Oz <<most important
Just-In-Time Runtime Optimization
Code Improvement OptionsConstant foldingx=3+4*5;x = 23;Constant propagationn=3;b=y*n+n;b=y*3+3;Assign variables to registers in C/C++register int x,y;Operator strength reductionx=y*3;x=y+y+y;Peephole optimization (use architecture-specific instructions)a += 1;
Compiler option to target different architectures (386, 486, Pentium, i5)Aligning data structures on natural boundaries (unaligned data accesses fault on some and are slower on all)
Common sub-expression optimizationx=(n+2)*y;y=z/(n+2)t=(n+2)then use tInline functions(treat function definition as a macro and substitute the text at every call)
Invariant code motion out of loopswhile (x++<Y) { z += p+n*6; p--;}n*6 never changesLoop fusion (make one loop out of two or more
(i.e. omp collapse))Loop unrolling (reduce iteration by factor of n,
replicate loop body n times)Loop interchange (change nesting order of loops,
which may enable other optimizations)Loop blocking or tiling (replace array processing
by two loops to divide the iteration space into smaller blocks to minimize cache misses
Omit frame pointers(procedure entry-exit code can be simplified when procedure call chain is deterministic)
Max Function#include <stdio.h>#include <stdlib.h>#include <time.h>#define N 20000000
int array_int_max(int a[], int n) { int i, max=0; for (i=1; i<n; i++) if (a[max]<a[i]) max=i; return max;}int test[N];int main(int argc, char *argv[]) { int i, j;for (i=0; i<N; i++) test[i]=rand();j=clock(); i=array_int_max(test,N);printf("clock=%ld index=%d max=%d\n", clock()-j, i, test[i]);return 0;}OUTPUTclock=96782 index=1310 max=2147483531
Microsoft Visual Studio Timings for Max Function
Build Options Clock() TimingDebug 131
Release, Optimization disabled,Favor small code, no whole program optimization 127
Release, Minimize size, Favor small code,no whole program optimization 42
Release, Minimize size, Favor fast code,no whole program optimization 29
Release, Minimize size, Favor fast code,Whole program optimization 29
Release, Maximize speed, Favor fast code,Whole program optimization 31#pragma omp sections, 2 threads 37
Pipeline Hazards
• Structural hazard– hardware resource conflicts prevent overlapped
execution.
• Control hazard– when any instruction, such as a branch, changes the
instruction pointer register (IP). The choices are to stall after a branch IF, to undo un-branched-to instructions, or to predict where every branch is going.
• Data hazard– An instruction produces output or an action that is
needed by a later instruction’s pipeline stage
Pipeline Optimization
Loop Unrolling
int A[N][N], B[N][N], C[N][N];int main(int argc, char *argv[]) {int i, j, k, z;for (i=0; i<N; i++) for (j=0; j<N; j++) { A[i][j]=rand(); B[i][j]=A[i][j]+1; C[i][j]=A[i][j]-1;}z=clock();for (i=0; i<N; i+=4) //increment by unrolling factorfor (j=0; j<N; j++)for (k=0; k<N; k++) { //8301 clocks, no unrolling A[i][j] = A[i][j] + B[i][k] * C[k][j]; //4281 clocks, 2 statements A[i+1][j] = A[i+1][j] + B[i+1][k] * C[k][j]; //3251 clocks, 3 statements A[i+2][j] = A[i+2][j] + B[i+2][k] * C[k][j]; //3063 clocks, 4 statements A[i+3][j] = A[i+3][j] + B[i+3][k] * C[k][j];}printf("clock=%d\n", clock()-z);return 0;}
Software Pipelining• Loop over statements, each statement is
dependent on the previous statement.
– ai, bi, ci
• Loop unrolling would result in
– ai, bi, ci, ai+1, bi+1, ci+1
• However, the dependency (data hazard) between b and a and between c and b still exist.
• Software pipelining changes loop to contain
– ai, ai+1, bi, bi+1, ci, ci+1
Vector Instruction Data Types
Vector Instruction Processing
Intel SSE
Vector Max
Function
#define N 20000000int array_int_max(vInt32 a[], int n) {int i; vInt32 max, temp, temp1;vCopy(max, a[0]);for (i=1; i<n; i+=4) { vMax_int(temp,a[i],a[i+1]); vMax_int(temp1,a[i+2],a[i+3]); vMax_int(max,temp,max); vMax_int(max,temp1,max);}vSplat_int(temp,max,0); vMax_int(max,temp,max);vSplat_int(temp,max,1); vMax_int(max,temp,max);vSplat_int(temp,max,2); vMax_int(max,temp,max);return vExtract_int(max,3);}
int test[N];int main(int argc, char *argv[]) {int i, j;for (i=0; i<N; i++) test[i]=rand();j=clock();i=array_int_max((vInt32 *) test, N/4);printf("clock=%d max=%d\n", clock()-j, i);}
a[0] a[1] a[2] a[3]0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
vCopy(max,a[0]);max temp temp10 1 2 3
vMax_int(temp,a[i],a[i+1]); 8 9 10 11vMax_int(temp1,a[i+2],a[i+3]); 12 13 14 15vMax_int(max,temp,max); 8 9 10 11vMax_int(max,temp1,max); 12 13 14 15vSplat_int(temp,max,0); 12 12 12 12vMax_int(max,temp,max); 12 13 14 15vSplat_int(temp,max,0); 13 13 13 13vMax_int(max,temp,max); 13 13 14 15vSplat_int(temp,max,0); 14 14 14 14vMax_int(max,temp,max); 14 14 14 15return vExtract_int(max,3); 15
CPU versus GPU Parallelism
Nvidia Tesla GPU
OpenCL Execution Model• Context
– Defines the target execution environment for a Program. A Context can include muliple GPUs and a CPU.
• Kernel– A C-like method executed on a streaming processor (also referred to as a
processing element).
– Kernel code only uses registers, no stack and no heap. Kernel code that uses more registers than are available may fail to load or execute inefficiently.
– No nested kernel calls, no recursion.
– Kernels are compiled for every device in a context.
• Kernel Arguments– Scalar
– Vector (128 bits, 4 floats or ints, 2 doubles)
– Pointer to a 1-d sequence of values no matter what the shape of the data.
• Program– Collection of kernels. Must be dynamically loaded into one or more
CPU/GPUs.
OpenCL GPU Storage Model
MemoryAccess Speed
VisibilityGPU Access
Host Access
Private Faster Work Item Read/Write NoneLocal Faster Work Group Read/Write NoneConstant Slower NDRange Read WriteGlobal Slower NDRange Read/write Read/Write
OpenCL Vector Additionconst char * sProgramSource =
"__kernel void vectorAdd( \n" \
"__global const float * a, \n" \
"__global const float * b, \n" \
"__global float * c) \n" \
"{ \n" \
" // Vector element index \n" \
" int nIndex = get_global_id(0); \n" \
" c[nIndex] = a[nIndex] + b[nIndex]; \n" \
"} \n";
OpenCL Vector Addition
• No use or private or local storage
• Reference to __global is slow
• Computation per PE is too little