INTEL CONFIDENTIAL Shared Memory Considerations Introduction to Parallel Programming – Part 4.
-
Upload
briana-perry -
Category
Documents
-
view
230 -
download
1
Transcript of INTEL CONFIDENTIAL Shared Memory Considerations Introduction to Parallel Programming – Part 4.
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Review & Objectives
Previously:Described method to identify opportunities for parallelism in
code segments and applications
At the end of this part you should be able to: Describe the shared-memory model of parallel
programmingDemonstrate how to implement domain and
task decompositions using threadsDecide whether a variable in a multithreaded
program should be shared or private
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Cooperating Parallel Processes
Parallel computing multiple processes working together to speed the solution of a task
Working together process cooperationKinds of cooperation
Sharing information (communication)Keeping out of each other’s way
(synchronization)
3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Shared-Memory Model
4
Shared Memory
Core
PrivateMemory
Core
PrivateMemory
Core
PrivateMemory
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Evaluating Parallel Models
How do processes share information?How do processes synchronize?In shared-memory model, both accomplished
through shared variablesCommunication: bufferSynchronization: semaphore
5
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Methodology
Study problem, sequential program, or code segmentLook for opportunities for parallelismUse threads to express parallelism
6
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
What Is a Process?
A program in some state of executionCodeDataLogical address space
Information about a process includesProcess stateProgram counterValues of core’s registersMemory management information
7
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
What Is a Thread?
“A unit of control within a process” — Carver and Tai
Main thread executes program’s “main” functionMain thread may create other threads to execute
other functionsThreads have own program counter, copy of core
registers, and stack of activation recordsThreads share process’s data, code, address space,
and other resourcesThreads have lower overhead than processes
8
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
Utility of Threads
Threads are flexible enough to implementDomain decompositionTask decompositionPipelining
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition Using Threads
10
SharedMemory
Thread 0 Thread 2
Thread 1
f ( )
f ( )
f ( )
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition Using Threads
11
SharedMemory
Thread 0 Thread 1
e ( )
g ( )h ( )
f ( )
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Pipelining Using Threads
12
Thread 0 Thread 2Thread 1
Shared Memory
Input Output
e ( ) f ( ) g ( )
Dataset 2
Data sets5, 6, ...
Dataset 4 Data
set 3 Dataset 1
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Shared versus Private Variables
13
SharedVariables
PrivateVariables
PrivateVariables
Thread
Thread
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);
14
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);
15
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Domain Decomposition
Sequential Code:int a[1000], i;for (i = 0; i < 1000; i++) a[i] = foo(i);Thread 0:for (i = 0; i < 500; i++) a[i] = foo(i);Thread 1:for (i = 500; i < 1000; i++) a[i] = foo(i);
16
SharedPrivate
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k); ...}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
17
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
18
Thread 0
Thread 1
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
19
Thread 0
Thread 1
Static variable: Shared
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
20
Thread 0
Thread 1
Heap variable: Shared
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Task Decomposition
Volatile int e;
main () { int x[10], j, k, m; j = f(x, k); m = g(x, k);}
int f(int *x, int k){ int a; a = e * x[k] * x[k]; return a;}
int g(int *x, int k){ int a; k = k-1; a = e / x[k]; return a;}
21
Thread 0
Thread 1
Function’s local variables: Private
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
Shared and Private Variables
Shared variablesStatic variablesHeap variablesContents of run-time stack at time of call
Private variablesLoop index variablesRun-time stack of functions invoked by thread
22
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Shared-Memory Model (Reprise)
23
Shared Memory
Core
PrivateMemory
Core
PrivateMemory
Core
PrivateMemory
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
The Threads Model
24
Shared Variables
Thread
PrivateVariables
Thread
PrivateVariables
Thread
PrivateVariables
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
References
Jim Beveridge and Robert Wiener, Multithreading Applications in Win32®, Addison-Wesley (1997).
David R. Butenhof, Programming with POSIX® Threads, Addison-Wesley, (1997).
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
25