Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo...
-
Upload
raymond-darcy-thompson -
Category
Documents
-
view
220 -
download
0
description
Transcript of Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo...
![Page 1: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/1.jpg)
Data-Parallel Programmingusing SaClecture 3
F21DP Distributed and Parallel Technology
Sven-Bodo Scholz
![Page 2: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/2.jpg)
2
Where do these Operations Come from?
double l2norm( double[*] A){ return( sqrt( sum( square( A)));}
double square( double A){ return( A*A);}
![Page 3: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/3.jpg)
3
Where do these Operations Come from?
double square( double A){ return( A*A);}
double[+] square( double[+] A){ res = with { (. <= iv <= .) : square( A[iv]); } : modarray( A); return( res);}
![Page 4: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/4.jpg)
4
With-Loopswith { ([0,0] <= iv < [3,4]) : square( iv[0]);} : genarray( [3,4], 42);
[0,0] [0,1] [0,2] [0,3]
[1,0] [1,1] [1,2] [1,3]
[2,0] [2,1] [2,2] [2,3]
0 0 0 0
1 1 1 1
4 4 4 4
indices values
![Page 5: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/5.jpg)
5
With-Loopswith { ([0,0] <= iv <= [1,1]) : square( iv[0]); ([0,2] <= iv <= [1,3]) : 42; ([2,0] <= iv <= [2,2]) : 0;} : genarray( [3,4], 21);
[0,0] [0,1] [0,2] [0,3]
[1,0] [1,1] [1,2] [1,3]
[2,0] [2,1] [2,2] [2,3]
0 0 42 42
1 1 42 42
0 0 0 21
indices values
![Page 6: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/6.jpg)
6
With-Loopswith { ([0,0] <= iv <= [1,1]) : square( iv[0]); ([0,2] <= iv <= [1,3]) : 42; ([2,0] <= iv <= [2,3]) : 0;} : fold( +, 0);
[0,0] [0,1] [0,2] [0,3]
[1,0] [1,1] [1,2] [1,3]
[2,0] [2,1] [2,2] [2,3]
0 0 42 42
1 1 42 42
0 0 0 0
indices values
map reduce
170
![Page 7: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/7.jpg)
7
Set-Notation and With-Loops
{ iv -> a[iv] + 1}
with { ( 0*shape(a) <= iv < shape(a)) : a[iv] + 1;} : genarray( shape( a), zero(a))
![Page 8: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/8.jpg)
Observation
most operations boil down to With-loops With-Loops are the source of concurrency
![Page 9: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/9.jpg)
9
Modules and NamespacesModule A;export all;
int foo(){...}
int bar(){...}
int main(){ x = A::foo(); y = A::bar(); ...}
Defines a namespace “A”
int foo(){...}
int main(){ x = A::foo(); y = A::bar(); z = foo(); ...}
lives in “MAIN”
refers to “A”
![Page 10: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/10.jpg)
10
Using Modules...Module A;export all;
int foo(){...}
int bar(){...}
use A: all;
int main(){ x = foo(); y = bar(); ...}
use A: all;
int foo(){...}
int main(){ x = A::foo(); y = bar(); z = foo(); ...}
makes all symbols of “A” directly usable in
“MAIN”
use A: all except {foo};
int foo(){...}
int main(){ x = A::foo(); y = bar(); z = foo(); ...}
![Page 11: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/11.jpg)
11
Modules and OverloadingModule A;export all;
int foo( int[*] a){...}
import A: all;
int foo( int[.] v){...}
int main(){ ... x = foo( y); ...}
?
literally imports all definitions from “A”
Module B;provide all;
int foo( int[*] a){...}
inhibits imports but allows uses
![Page 12: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/12.jpg)
12
States in SaCClass stack;classtype int[100];export all;
stack createStack(){...}stack push( stack s, int val){ ...}
use stack: all;
int main(){ S = createStack(); S = push( S, 10); S = push( S, 42); ...}
use stack: all;
int main(){ S = createStack(); S1 = push( S, 10); S2 = push( S, 42); ...}
introduces new type “stack”
defines the representation of the type “stack”
can be done destructively!
![Page 13: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/13.jpg)
13
Reference ParametersClass stack;classtype int[100];export all;
stack createStack(){...}void push( stack &s, int val){ ...}
use stack: all;
int main(){ S = createStack(); push( S, 10); push( S, 42); ...}
use stack: all;
int main(){ S = createStack(); S = push( S, 10); S = push( S, 42); ...}
declares that a modified version of s is returned
internally transformed
![Page 14: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/14.jpg)
14
Global Objects!!Class stack;classtype int[100];export all;
objdef stack myS= createStack();stack createStack(){...}void push(int val){ ...myS...}
use stack: all;
int main(){ push( 10); push( 42); ...}
use stack: all;
int main(){ myS = createStack(); myS = push( myS, 10); myS = push( myS, 42); ...}
internally transformed
![Page 15: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/15.jpg)
15
Asynchronous I/O?
• scales with #cores• improves debugging • enables visualisation
![Page 16: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/16.jpg)
16
But how to model the data dependencies?
![Page 17: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/17.jpg)
17
Attempt #1: Split State
![Page 18: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/18.jpg)
18
Attempt #1: Split State
may not be possible!
![Page 19: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/19.jpg)
19
Solution: Non-Deterministic Order!
![Page 20: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/20.jpg)
20
Practical Consequence
with { ([0] <= [i] <[10]) { printf( “Hi, I am # %d\n”, i); }} : void
is legal SaC!!
![Page 21: Data-Parallel Programming using SaC lecture 3 F21DP Distributed and Parallel Technology Sven-Bodo Scholz.](https://reader036.fdocuments.in/reader036/viewer/2022062505/5a4d1bdd7f8b9ab0599ddb7d/html5/thumbnails/21.jpg)
21
Practical Consequence
with { ([0] <= [i] <[10]) { printf( “Hi, I am # %d\n”, i); }} : void ;
translates into
stdout = with { ([0] <= [i] <[10]) { stdout = printf( stdout, “Hi, I am # %d\n”, i); } : stdout; } : propagate( stdout);