CS1760 Multiprocessor Synchronization - Brown...
Transcript of CS1760 Multiprocessor Synchronization - Brown...
![Page 1: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/1.jpg)
CS1760
Multiprocessor
Synchronization
![Page 2: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/2.jpg)
Staff
Art of Multiprocessor Programming
Maurice Herlihy (instructor)
Bhrath Kayyer (UTA)
Daniel Engel (grad TA)
Jonathan Lister (HTA)
![Page 3: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/3.jpg)
Grading
Art of Multiprocessor Programming
8 Homeworks (40%)
3 Midterms (40%)
5 programming assignments (20%)
![Page 4: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/4.jpg)
Collaboration
Art of Multiprocessor Programming
talking about the homework
problems with other students; using
other textbooks; using the Internet.
obtaining the answer directly from
anyone or anything else in any
form.
Permitted
NOT Permitted
![Page 5: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/5.jpg)
Capstone
Art of Multiprocessor Programming
Yes, you can take this course as a
capstone course
Only one project possible
(concurrent packet filter)
Requires reading ahead of the course
See web page for details
![Page 6: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/6.jpg)
See Course Web Page for …
Art of Multiprocessor Programming
Piazza
TA Hours
Other important matters
![Page 7: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/7.jpg)
Art of Multiprocessor Programming 7
Moore’s Law
Clock
speed
flattening
sharply
Transistor
count still
rising
![Page 8: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/8.jpg)
Moore’s Law (in practice)
Art of Multiprocessor Programming 8
![Page 9: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/9.jpg)
Art of Multiprocessor Programming 9
Extinct: the Uniprocesor
memory
cpu
![Page 10: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/10.jpg)
Art of Multiprocessor Programming 10
Extinct:
The Shared Memory Multiprocessor
(SMP)
cache
BusBus
shared memory
cachecache
![Page 11: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/11.jpg)
Art of Multiprocessor Programming 11
The New Boss:
The Multicore Processor
(CMP)
cache
BusBus
shared memory
cachecacheAll on the
same chip
Sun
T2000
Niagara
![Page 12: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/12.jpg)
Art of Multiprocessor Programming 12
From the 2008 press…
…Intel has announced a press conference in
San Francisco on November 17th, where it
will officially launch the Core i7 Nehalem
processor…
…Sun’s next generation Enterprise T5140
and T5240 servers, based on the 3rd
Generation UltraSPARC T2 Plus processor,
were released two days ago…
![Page 13: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/13.jpg)
Art of Multiprocessor Programming 13
Why is Kunle Smiling?
Niagara 1
![Page 14: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/14.jpg)
Art of Multiprocessor Programming 14
Why do we care?
• Time no longer cures software bloat
– The “free ride” is over
• When you double your program’s path
length
– You can’t just wait 6 months
– Your software must somehow exploit twice as
much concurrency
![Page 15: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/15.jpg)
Art of Multiprocessor Programming 15
Traditional Scaling Process
User code
Traditional
Uniprocessor
Speedup
1.8x
7x
3.6x
Time: Moore’s law
![Page 16: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/16.jpg)
Ideal Scaling Process
Art of Multiprocessor Programming 16
User code
Multicore
Speedup 1.8x
7x
3.6x
Unfortunately, not so simple…
![Page 17: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/17.jpg)
Actual Scaling Process
Art of Multiprocessor Programming 17
1.8x 2x 2.9x
User code
Multicore
Speedup
Parallelization and Synchronization
require great care…
![Page 18: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/18.jpg)
Art of Multiprocessor Programming 18
Multicore Programming:
Course Overview
• Fundamentals
– Models, algorithms, impossibility
• Real-World programming
– Architectures
– Techniques
![Page 19: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/19.jpg)
Art of Multiprocessor Programming 19
Sequential Computation
memory
object object
thread
![Page 20: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/20.jpg)
Art of Multiprocessor Programming 20
Concurrent Computation
memory
object object
![Page 21: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/21.jpg)
Art of Multiprocessor Programming 21
Asynchrony
• Sudden unpredictable delays
– Cache misses (short)
– Page faults (long)
– Scheduling quantum used up (really long)
![Page 22: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/22.jpg)
Art of Multiprocessor Programming 22
Model Summary
• Multiple threads
– Sometimes called processes
• Single shared memory
• Objects live in memory
• Unpredictable asynchronous delays
![Page 23: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/23.jpg)
23
Road Map
• We are going to focus on principles first,
then practice
– Start with idealized models
– Look at simplistic problems
– Emphasize correctness over pragmatism
– “Correctness may be theoretical, but
incorrectness has practical impact”
Art of Multiprocessor Programming
![Page 24: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/24.jpg)
24
Concurrency Jargon
• Hardware
– Processors
• Software
– Threads, processes
• Sometimes OK to confuse them,
sometimes not.
Art of Multiprocessor Programming
![Page 25: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/25.jpg)
25
Parallel Primality Testing
• Challenge
– Print primes from 1 to 1010
• Given
– Ten-processor multiprocessor
– One thread per processor
• Goal
– Get ten-fold speedup (or close)
Art of Multiprocessor Programming
![Page 26: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/26.jpg)
Art of Multiprocessor Programming 26
Load Balancing
• Split the work evenly
• Each thread tests range of 109
…
…109 10102·1091
P0 P1 P9
![Page 27: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/27.jpg)
27
Procedure for Thread i
void primePrint {
int i = ThreadID.get(); // IDs in {0..9}
for (j = i*109+1, j<(i+1)*109; j++) {
if (isPrime(j))
print(j);
}
}
Art of Multiprocessor Programming
![Page 28: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/28.jpg)
28
Issues
• Higher ranges have fewer primes
• Yet larger numbers harder to test
• Thread workloads
– Uneven
– Hard to predict
Art of Multiprocessor Programming
![Page 29: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/29.jpg)
Art of Multiprocessor Programming 29
Issues
• Higher ranges have fewer primes
• Yet larger numbers harder to test
• Thread workloads
– Uneven
– Hard to predict
• Need dynamic load balancing
![Page 30: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/30.jpg)
Art of Multiprocessor Programming 30
17
18
19
Shared Counter
each thread
takes a number
![Page 31: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/31.jpg)
31
Procedure for Thread i
int counter = new Counter(1);
void primePrint {
long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}
Art of Multiprocessor Programming
![Page 32: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/32.jpg)
Art of Multiprocessor Programming 32
Counter counter = new Counter(1);
void primePrint {
long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}
Procedure for Thread i
Shared counter
object
![Page 33: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/33.jpg)
Art of Multiprocessor Programming 33
Where Things Reside
cache
BusBus
cachecache
1
shared counter
shared
memory
void primePrint {
int i =
ThreadID.get(); // IDs
in {0..9}
for (j = i*109+1,
j<(i+1)*109; j++) {
if (isPrime(j))
print(j);
}
}
code
Local
variables
![Page 34: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/34.jpg)
Art of Multiprocessor Programming 34
Procedure for Thread i
Counter counter = new Counter(1);
void primePrint {
long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}
Stop when every
value taken
![Page 35: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/35.jpg)
Art of Multiprocessor Programming 35
Counter counter = new Counter(1);
void primePrint {
long j = 0;
while (j < 1010) {
j = counter.getAndIncrement();
if (isPrime(j))
print(j);
}
}
Procedure for Thread i
Increment & return each
new value
![Page 36: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/36.jpg)
36
Counter Implementation
public class Counter {
private long value;
public long getAndIncrement() {
return value++;
}
}
Art of Multiprocessor Programming
![Page 37: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/37.jpg)
Art of Multiprocessor Programming 37
Counter Implementation
public class Counter {
private long value;
public long getAndIncrement() {
return value++;
}
}
![Page 38: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/38.jpg)
Art of Multiprocessor Programming 38
What It Means
public class Counter {
private long value;
public long getAndIncrement() {
return value++;
}
}
![Page 39: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/39.jpg)
Art of Multiprocessor Programming 39
What It Means
public class Counter {
private long value;
public long getAndIncrement() {
return value++;
}
}
temp = value;
value = temp + 1;
return temp;
![Page 40: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/40.jpg)
Art of Multiprocessor Programming 40
time
Not so good…
Value… 1
read
1
read
1
write
2
read
2
write
3
write
2
2 3 2
![Page 41: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/41.jpg)
Art of Multiprocessor Programming 41
Is this problem inherent?
If we could only glue reads and writes
together…
read
write read
write
!! !!
![Page 42: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/42.jpg)
42
Challenge
public class Counter {
private long value;
public long getAndIncrement() {
temp = value;
value = temp + 1;
return temp;
}
}
Art of Multiprocessor Programming
![Page 43: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/43.jpg)
Art of Multiprocessor Programming 43
Challenge
public class Counter {
private long value;
public long getAndIncrement() {
temp = value;
value = temp + 1;
return temp;
}
}
Make these steps
atomic (indivisible)
![Page 44: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/44.jpg)
Art of Multiprocessor Programming 44
Hardware Solution
public class Counter {
private long value;
public long getAndIncrement() {
temp = value;
value = temp + 1;
return temp;
}
} ReadModifyWrite()
instruction
![Page 45: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/45.jpg)
Art of Multiprocessor Programming 45
An Aside: Java™
public class Counter {
private long value;
public long getAndIncrement() {
synchronized {
temp = value;
value = temp + 1;
}
return temp;
}
}
![Page 46: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/46.jpg)
Art of Multiprocessor Programming 46
An Aside: Java™
public class Counter {
private long value;
public long getAndIncrement() {
synchronized {
temp = value;
value = temp + 1;
}
return temp;
}
}Synchronized block
![Page 47: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/47.jpg)
Art of Multiprocessor Programming 47
An Aside: Java™
public class Counter {
private long value;
public long getAndIncrement() {
synchronized {
temp = value;
value = temp + 1;
}
return temp;
}
}
Mutual Exclusion
![Page 48: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/48.jpg)
48
Mutual Exclusion,
or “Alice & Bob share a pond”
A B
Art of Multiprocessor Programming
![Page 49: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/49.jpg)
49
Alice has a pet
A B
Art of Multiprocessor Programming
![Page 50: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/50.jpg)
50
Bob has a pet
A B
Art of Multiprocessor Programming
![Page 51: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/51.jpg)
51
The Problem
A B
The pets don’t
get along
Art of Multiprocessor Programming
![Page 52: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/52.jpg)
52
Formalizing the Problem
• Two types of formal properties in
asynchronous computation:
• Safety Properties
– Nothing bad happens ever
• Liveness Properties
– Something good happens eventually
Art of Multiprocessor Programming
![Page 53: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/53.jpg)
53
Formalizing our Problem
• Mutual Exclusion
– Both pets never in pond simultaneously
– This is a safety property
• No Deadlock
– if only one wants in, it gets in
– if both want in, one gets in.
– This is a liveness property
Art of Multiprocessor Programming
![Page 54: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/54.jpg)
54
Simple Protocol
• Idea
– Just look at the pond
• Gotcha
– Not atomic
– Trees obscure the view
Art of Multiprocessor Programming
![Page 55: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/55.jpg)
55
Interpretation
• Threads can’t “see” what other threads are
doing
• Explicit communication required for
coordination
Art of Multiprocessor Programming
![Page 56: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/56.jpg)
56
Cell Phone Protocol
• Idea
– Bob calls Alice (or vice-versa)
• Gotcha
– Bob takes shower
– Alice recharges battery
– Bob out shopping for pet food …
Art of Multiprocessor Programming
![Page 57: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/57.jpg)
57
Interpretation
• Message-passing doesn’t work
• Recipient might not be
– Listening
– There at all
• Communication must be
– Persistent (like writing)
– Not transient (like speaking)
Art of Multiprocessor Programming
![Page 58: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/58.jpg)
58
Can Protocol
co
la
cola
Art of Multiprocessor Programming
![Page 59: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/59.jpg)
59
Bob conveys a bit
A B
co
la
Art of Multiprocessor Programming
![Page 60: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/60.jpg)
60
Bob conveys a bit
A B
Art of Multiprocessor Programming
![Page 61: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/61.jpg)
61
Can Protocol
• Idea
– Cans on Alice’s windowsill
– Strings lead to Bob’s house
– Bob pulls strings, knocks over cans
• Gotcha
– Cans cannot be reused
– Bob runs out of cans
Art of Multiprocessor Programming
![Page 62: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/62.jpg)
62
Interpretation
• Cannot solve mutual exclusion with
interrupts
– Sender sets fixed bit in receiver’s space
– Receiver resets bit when ready
– Requires unbounded number of interrupt bits
Art of Multiprocessor Programming
![Page 63: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/63.jpg)
63
Flag Protocol
A B
Art of Multiprocessor Programming
![Page 64: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/64.jpg)
64
Alice’s Protocol (sort of)
A B
Art of Multiprocessor Programming
![Page 65: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/65.jpg)
65
Bob’s Protocol (sort of)
A B
Art of Multiprocessor Programming
![Page 66: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/66.jpg)
66
Alice’s Protocol
• Raise flag
• Wait until Bob’s flag is down
• Unleash pet
• Lower flag when pet returns
Art of Multiprocessor Programming
![Page 67: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/67.jpg)
Art of Multiprocessor Programming 67
Bob’s Protocol
• Raise flag
• Wait until Alice’s flag is down
• Unleash pet
• Lower flag when pet returns
![Page 68: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/68.jpg)
68
Bob’s Protocol (2nd try)
• Raise flag
• While Alice’s flag is up
– Lower flag
– Wait for Alice’s flag to go down
– Raise flag
• Unleash pet
• Lower flag when pet returns
Art of Multiprocessor Programming
![Page 69: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/69.jpg)
Art of Multiprocessor Programming 69
Bob’s Protocol
• Raise flag
• While Alice’s flag is up
– Lower flag
– Wait for Alice’s flag to go down
– Raise flag
• Unleash pet
• Lower flag when pet returns
Bob defers
to Alice
![Page 70: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/70.jpg)
70
The Flag Principle
• Raise the flag
• Look at other’s flag
• Flag Principle:
– If each raises and looks, then
– Last to look must see both flags up
Art of Multiprocessor Programming
![Page 71: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/71.jpg)
71
Proof of Mutual Exclusion
• Assume both pets in pond
– Derive a contradiction
– By reasoning backwards
• Consider the last time Alice and Bob each
looked before letting the pets in
• Without loss of generality assume Alice
was the last to look…
Art of Multiprocessor Programming
![Page 72: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/72.jpg)
Art of Multiprocessor Programming 72
Proof
time
Alice’s last look
Alice last raised her flag
Bob’s last
look
Alice must have seen Bob’s Flag. A Contradiction
Bob last raised
flag
![Page 73: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/73.jpg)
73
Proof of No Deadlock
• If only one pet wants in, it gets in.
Art of Multiprocessor Programming
![Page 74: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/74.jpg)
74
Proof of No Deadlock
• If only one pet wants in, it gets in.
• Deadlock requires both continually trying
to get in.
Art of Multiprocessor Programming
![Page 75: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/75.jpg)
Art of Multiprocessor Programming 75
Proof of No Deadlock
• If only one pet wants in, it gets in.
• Deadlock requires both continually trying
to get in.
• If Bob sees Alice’s flag, he backs off, gives
her priority (Alice’s lexicographic privilege)
![Page 76: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/76.jpg)
76
Remarks
• Protocol is unfair
– Bob’s pet might never get in
• Protocol uses waiting
– If Bob is eaten by his pet, Alice’s pet might
never get in
Art of Multiprocessor Programming
![Page 77: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/77.jpg)
77
Moral of Story
• Mutual Exclusion cannot be solved by
–transient communication (cell phones)
– interrupts (cans)
• It can be solved by
– one-bit shared variables
– that can be read or written
Art of Multiprocessor Programming
![Page 78: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/78.jpg)
Art of Multiprocessor Programming 78
The Arbiter Problem (an aside)
Pick a
point
Pick a
point
![Page 79: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/79.jpg)
79
The Fable Continues
• Alice and Bob fall in love & marry
Art of Multiprocessor Programming
![Page 80: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/80.jpg)
80
The Fable Continues
• Alice and Bob fall in love & marry
• Then they fall out of love & divorce
– After a coin flip, she gets the pets
– He has to feed them
Art of Multiprocessor Programming
![Page 81: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/81.jpg)
81
The Fable Continues
• Alice and Bob fall in love & marry
• Then they fall out of love & divorce
– She gets the pets
– He has to feed them
• Leading to a new coordination problem:
Producer-Consumer
Art of Multiprocessor Programming
![Page 82: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/82.jpg)
82
Bob Puts Food in the Pond
A
Art of Multiprocessor Programming
![Page 83: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/83.jpg)
83
mmm…
Alice releases her pets to Feed
Bmmm…
Art of Multiprocessor Programming
![Page 84: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/84.jpg)
84
Producer/Consumer
• Alice and Bob can’t meet
– Each has restraining order on other
– So he puts food in the pond
– And later, she releases the pets
• Avoid
– Releasing pets when there’s no food
– Putting out food if uneaten food remains
Art of Multiprocessor Programming
![Page 85: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/85.jpg)
85
Producer/Consumer
• Need a mechanism so that
– Bob lets Alice know when food has been put
out
– Alice lets Bob know when to put out more
food
Art of Multiprocessor Programming
![Page 86: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/86.jpg)
86
Surprise Solution
A B
co
la
Art of Multiprocessor Programming
![Page 87: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/87.jpg)
87
Bob puts food in Pond
A B
co
la
Art of Multiprocessor Programming
![Page 88: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/88.jpg)
88
Bob knocks over Can
A B
Art of Multiprocessor Programming
![Page 89: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/89.jpg)
89
Alice Releases Pets
A Byum… Byum…
Art of Multiprocessor Programming
![Page 90: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/90.jpg)
90
Alice Resets Can when Pets are
Fed
A B
co
la
Art of Multiprocessor Programming
![Page 91: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/91.jpg)
Art of Multiprocessor Programming 91
Pseudocode
while (true) {
while (can.isUp()){};
pet.release();
pet.recapture();
can.reset();
}
Alice’s code
![Page 92: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/92.jpg)
Art of Multiprocessor Programming 92
Pseudocode
while (true) {
while (can.isUp()){};
pet.release();
pet.recapture();
can.reset();
}
Alice’s code
while (true) {
while (can.isDown()){};
pond.stockWithFood();
can.knockOver();
}
Bob’s code
![Page 93: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/93.jpg)
93
Correctness
• Mutual Exclusion
– Pets and Bob never together in pond
Art of Multiprocessor Programming
![Page 94: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/94.jpg)
94
Correctness
• Mutual Exclusion
– Pets and Bob never together in pond
• No Starvation
if Bob always willing to feed, and pets always
famished, then pets eat infinitely often.
Art of Multiprocessor Programming
![Page 95: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/95.jpg)
Art of Multiprocessor Programming 95
Correctness
• Mutual Exclusion
– Pets and Bob never together in pond
• No Starvation
if Bob always willing to feed, and pets always famished, then pets eat infinitely often.
• Producer/Consumer
The pets never enter pond unless there is food, and Bob never provides food if there is unconsumed food.
safety
liveness
safety
![Page 96: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/96.jpg)
96
Could Also Solve Using Flags
A B
Art of Multiprocessor Programming
![Page 97: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/97.jpg)
97
Waiting
• Both solutions use waiting
– while(mumble){}
• In some cases waiting is problematic
– If one participant is delayed
– So is everyone else
– But delays are common & unpredictable
Art of Multiprocessor Programming
![Page 98: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/98.jpg)
98
The Fable drags on …
• Bob and Alice still have issues
Art of Multiprocessor Programming
![Page 99: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/99.jpg)
99
The Fable drags on …
• Bob and Alice still have issues
• So they need to communicate
Art of Multiprocessor Programming
![Page 100: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/100.jpg)
100
The Fable drags on …
• Bob and Alice still have issues
• So they need to communicate
• They agree to use billboards …
Art of Multiprocessor Programming
![Page 101: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/101.jpg)
101
E1
D2
C3
Billboards are Large
B3A
1
Letter
TilesFrom Scrabble™ box
Art of Multiprocessor Programming
![Page 102: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/102.jpg)
102
E1
D2
C3
Write One Letter at a Time …
B3A
1
W4A
1S
1
H4
Art of Multiprocessor Programming
![Page 103: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/103.jpg)
103
To post a message
W4A
1S
1H
4A
1C
3R
1T
1H
4E
1
whew
Art of Multiprocessor Programming
![Page 104: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/104.jpg)
104
S1
Let’s send another message
S1E
1L
1L
1L
1V
4
L1 A
1
M3
A1
A1
P3
Art of Multiprocessor Programming
![Page 105: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/105.jpg)
105
Uh-Oh
A1
C3
R1
T1H
4E
1S
1E
1L
1L
1
L1
OK
Art of Multiprocessor Programming
![Page 106: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/106.jpg)
106
Readers/Writers
• Devise a protocol so that
– Writer writes one letter at a time
– Reader reads one letter at a time
– Reader sees “snapshot”
• Old message or new message
• No mixed messages
Art of Multiprocessor Programming
![Page 107: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/107.jpg)
107
Readers/Writers (continued)• Easy with mutual exclusion
• But mutual exclusion requires waiting
– One waits for the other
– Everyone executes sequentially
• Remarkably
– We can solve R/W without mutual exclusion
Art of Multiprocessor Programming
![Page 108: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/108.jpg)
Art of Multiprocessor Programming 108
Esoteric?
• Java container size() method
• Single shared counter?
– incremented with each add() and
– decremented with each remove()
• Threads wait to exclusively access counter
![Page 109: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/109.jpg)
109
Readers/Writers Solution
• Each thread i has size[i] counter
– only it increments or decrements.
• To get object’s size, a thread reads a
“snapshot” of all counters
• This eliminates the bottleneck
Art of Multiprocessor Programming
![Page 110: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/110.jpg)
110
Why do we care?
• We want as much of the code as possible
to execute concurrently (in parallel)
• A larger sequential part implies reduced
performance
• Amdahl’s law: this relation is not linear…
Art of Multiprocessor Programming
![Page 111: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/111.jpg)
Art of Multiprocessor Programming 111
Amdahl’s Law
Speedup=1-thread execution time
n-thread execution time
![Page 112: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/112.jpg)
Art of Multiprocessor Programming 112
Amdahl’s Law
Speedup=1
1 − 𝑝 +𝑝𝑛
![Page 113: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/113.jpg)
Parallel
fraction
Art of Multiprocessor Programming 113
Amdahl’s Law
Speedup=1
1 − 𝑝 +𝑝𝑛
![Page 114: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/114.jpg)
Art of Multiprocessor Programming 114
Amdahl’s Law
Speedup=
Sequential
fraction
1
1 − 𝑝 +𝑝𝑛
Parallel
fraction
![Page 115: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/115.jpg)
Art of Multiprocessor Programming 115
Amdahl’s Law
Speedup=
Parallel
fraction
Sequential
fraction
Number of
threads
1
1 − 𝑝 +𝑝𝑛
![Page 116: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/116.jpg)
Bad synchronization ruins everything
Amdal’s Law
![Page 117: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/117.jpg)
117
Example
• Ten processors
• 60% concurrent, 40% sequential
• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 118: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/118.jpg)
118
Example
• Ten processors
• 60% concurrent, 40% sequential
• How close to 10-fold speedup?
10
6.06.01
1
+−Speedup = 2.17=
Art of Multiprocessor Programming
![Page 119: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/119.jpg)
119
Example
• Ten processors
• 80% concurrent, 20% sequential
• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 120: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/120.jpg)
120
Example
• Ten processors
• 80% concurrent, 20% sequential
• How close to 10-fold speedup?
10
8.08.01
1
+−Speedup = 3.57=
Art of Multiprocessor Programming
![Page 121: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/121.jpg)
121
Example
• Ten processors
• 90% concurrent, 10% sequential
• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 122: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/122.jpg)
Art of Multiprocessor Programming 122
Example
• Ten processors
• 90% concurrent, 10% sequential
• How close to 10-fold speedup?
10
9.09.01
1
+−Speedup = 5.26=
![Page 123: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/123.jpg)
123
Example
• Ten processors
• 99% concurrent, 01% sequential
• How close to 10-fold speedup?
Art of Multiprocessor Programming
![Page 124: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/124.jpg)
Art of Multiprocessor Programming 124
Example
• Ten processors
• 99% concurrent, 01% sequential
• How close to 10-fold speedup?
10
99.099.01
1
+−Speedup = 9.17=
![Page 125: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/125.jpg)
Back to Real-World Multicore
Scaling
1.8x 2x 2.9x
User code
Multicore
Speedup
Not reducing
sequential % of code
Art of Multiprocessor Programming
![Page 126: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/126.jpg)
Shared Data Structures
75%
Unshared
25%
Shared
Coarse
Grained
Fine
Grained
75%
Unshared
25%
Shared
![Page 127: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/127.jpg)
Shared Data Structures
75%
Unshared
25%
Shared
Coarse
Grained
Fine
Grained
Why only 2.9 speedup
75%
Unshared
25%
Shared
Honk!
Honk!
Honk!
![Page 128: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/128.jpg)
Shared Data Structures
75%
Unshared
25%
Shared
Coarse
Grained
Fine
Grained
Why fine-grained
parallelism maters
75%
Unshared
25%
Shared
Honk!
Honk!
Honk!
![Page 129: CS1760 Multiprocessor Synchronization - Brown Universitycs.brown.edu/courses/cs176/lectures/chapter_01.pdf–By reasoning backwards • Consider the last time Alice and Bob each looked](https://reader035.fdocuments.in/reader035/viewer/2022071418/61163df314acb908921bdcec/html5/thumbnails/129.jpg)
Art of Multiprocessor Programming
Diminishing Returns
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
speedup
This course is about the parts that
are hard to make concurrent …
but still have a big influence on speedup!