Parallel Decomposition Methods
description
Transcript of Parallel Decomposition Methods
![Page 1: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/1.jpg)
INTEL CONFIDENTIAL
Parallel Decomposition MethodsIntroduction to Parallel Programming – Part 2
![Page 2: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/2.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Review & Objectives
Previously:Defined parallel computingExplained why parallel computing is becoming mainstreamExplained why explicit parallel programming is necessary
At the end of this part you should be able to:Identify opportunities for parallelism in code
segments and applicationsDescribe three methods of dividing independent
work
![Page 3: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/3.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
3
Methodology
Study problem, sequential program, or code segmentLook for opportunities for parallelismTry to keep all cores busy doing useful work
![Page 4: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/4.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
4
Ways of Exploiting Parallelism
Domain decompositionTask decompositionPipelining
![Page 5: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/5.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
5
Domain Decomposition
First, decide how data elements should be divided among cores
Second, decide which tasks each core should be doing
Example: Vector addition
![Page 6: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/6.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
6
Domain Decomposition
Find the largest element of an array
![Page 7: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/7.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
7
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 8: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/8.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
8
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 9: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/9.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 10: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/10.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
10
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 11: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/11.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
11
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 12: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/12.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
12
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 13: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/13.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
13
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 14: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/14.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
14
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 15: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/15.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
15
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 16: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/16.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
16
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 17: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/17.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
17
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
![Page 18: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/18.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
18
Task (Functional) Decomposition
First, divide problem into independent tasksSecond, decide which data elements are going to be
accessed (read and/or written) by which tasks
Example: Event-handler for GUI
![Page 19: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/19.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
19
Task Decomposition
s()
r()q()h()
g()
f()
![Page 20: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/20.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
20
Task Decomposition
s()
r()q()h()
g()
Core 0
Core 2
Core 1 f()
![Page 21: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/21.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
21
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 f() Core 0
![Page 22: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/22.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
22
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
![Page 23: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/23.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
23
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
![Page 24: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/24.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
24
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
![Page 25: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/25.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
25
Pipelining
Special kind of task decomposition“Assembly line” parallelismExample: 3D rendering in computer graphics
RasterizeClipProjectModel
Input Output
![Page 26: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/26.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
26
Processing One Data Set (Step 1)
RasterizeClipProjectModel
![Page 27: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/27.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
27
Processing One Data Set (Step 2)
RasterizeClipProjectModel
![Page 28: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/28.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
28
Processing One Data Set (Step 3)
RasterizeClipProjectModel
![Page 29: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/29.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
29
Processing One Data Set (Step 4)
RasterizeClipProjectModel
The pipeline processes 1 data set in 4 steps
![Page 30: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/30.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
30
Processing Two Data Sets (Step 1)
RasterizeClipProjectModel
![Page 31: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/31.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
31
Processing Two Data Sets (Time 2)
RasterizeClipProjectModel
![Page 32: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/32.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
32
Processing Two Data Sets (Step 3)
RasterizeClipProjectModel
![Page 33: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/33.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
33
Processing Two Data Sets (Step 4)
RasterizeClipProjectModel
![Page 34: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/34.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
34
Processing Two Data Sets (Step 5)
RasterizeClipProjectModel
The pipeline processes 2 data sets in 5 steps
![Page 35: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/35.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
35
Pipelining Five Data Sets (Step 1)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 36: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/36.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
36
Pipelining Five Data Sets (Step 2)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 37: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/37.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
37
Pipelining Five Data Sets (Step 3)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 38: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/38.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
38
Pipelining Five Data Sets (Step 4)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 39: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/39.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
39
Pipelining Five Data Sets (Step 5)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 40: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/40.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
40
Pipelining Five Data Sets (Step 6)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 41: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/41.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
41
Pipelining Five Data Sets (Step 7)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 42: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/42.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
42
Pipelining Five Data Sets (Step 8)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
![Page 43: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/43.jpg)
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
43
References
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005).
![Page 44: Parallel Decomposition Methods](https://reader035.fdocuments.in/reader035/viewer/2022062315/568166c9550346895ddad6cd/html5/thumbnails/44.jpg)